Today one colleague of mine asking me for tips on improving the DataTable.Select() performance. Apparently she has a data table which contains tens of thousands of records which some of the rows need to be processed together.
This particular question left me scratching my head. The only thing that I can suggest were:
- Make sure when populating the data table from the database, the data are already sorted
- Make sure the filter parameter of the Select() method is selecting by integer data
Come to think of it, I don’t really know how actually the Select() method is done. Is it using Binary Search, Linear Search, Interpolation Seach or any other search method that I never come across with.
If the case is when the data was fetched from database already in order of an ID, let say CustomerID and the DataTable.Select() is using “CustomerID=1001”, is there any way to optimize this further?
I’ll get back to you when I found the answer. Or maybe you have the answer?
GD Star Rating
loading...
DataTable.Select Optimization, loading...
About Hardono
Hi, I'm Hardono. I am working as a Software Developer. I am working mostly in Windows, dealing with .NET, conversing in C#. But I know a bit of Linux, mainly because I need to keep this blog operational. I've been working in Logistics/Transport industry for more than 9 years.
Incoming Search
datatable select performance
Did you know that there’s a way to legally “steal” traffic from
high-traffic websites and redirect it all to your website?
Well, I didn’t.
But, there’s a new free site that just launched today, and it
explains it all in plain English (for once)!
I’d *highly* recommend that you take a look at this if you’re
interested in driving a lot of free traffic to your website:
Exit Explosion
I know you’re itching to find out how it works…
edward
PS I contacted you because you are in blog roll. This is the chance that we could get more traffic. Have a nice day to you.
Ouch dude.. I thought someone gave me advice on the Selec() performance 🙂
Anyway, not really interested in killing my host though. This blog is hosted in a shared hosting with a cheap monthly fee. So driving thousands of people coming here would definitely melt-down the server =P
Thanks anyway…
DataTables are implemented internally with a BTree variation, just like most relational databases use. http://en.wikipedia.org/wiki/B-tree
Searching in from multiple angles and inherent in the structure. Usually, uou can speed things up by adding an index, though I don’t remember if DataTables support indexes. I know that searching on the primary key is incredibly fast compared to not searching on the key. So if you’re not using a PK, start using one.
Another way to speed things up is to process the rows into other tables so the search set is smaller, or to use DataViews to that effect.
Thanks for the insight, Xero.
But as for this case, my team ‘decided’ that traversing line-by-line is the best solution since the data is already sorted and we actually need to process each row.
As it turns out, we actually pointed the gun to the wrong suspect. After we run profiler, the bottleneck was not on the DataTable.Select(), but it was caused by invocation of web services.