DataTable.Select Optimization
Today one colleague of mine asking me for tips on improving the DataTable.Select() performance. Apparently she has a data table which contains tens of thousands of records which some of the rows need to be processed together.
This particular question left me dumbfounded and baffled. The only thing that I can suggest were:
- Make sure when populating the data table from the database, the data are already sorted
- Make sure the filter parameter of the Select() method is selecting by integer data
Come to think of it, I don’t really know how actually the Select() method is done. Is it using Binary Search, Linear Search, Interpolation Seach or any other search method that I never come across with.
If the case is when the data was fetched from database already in order of an ID, let say CustomerID and the DataTable.Select() is using “CustomerID=1001″, is there any way to optimize this further?
I’ll get back to you when I found the answer. Or maybe you have the answer?
Hardono Arifanto :: Aug.24.2007 :: Programming, .NET :: 5 Comments »
(No Ratings Yet)
Loading ...
5 Comments to “DataTable.Select Optimization”
-
edward
Posted: Aug 24th, 2007 at 3:50 am1Did you know that there’s a way to legally “steal” traffic from
high-traffic websites and redirect it all to your website?Well, I didn’t.
But, there’s a new free site that just launched today, and it
explains it all in plain English (for once)!I’d *highly* recommend that you take a look at this if you’re
interested in driving a lot of free traffic to your website:I know you’re itching to find out how it works…
edward
PS I contacted you because you are in blog roll. This is the chance that we could get more traffic. Have a nice day to you.
-
Hardono Arifanto
Posted: Aug 24th, 2007 at 9:36 pm2Ouch dude.. I thought someone gave me advice on the Selec() performance
Anyway, not really interested in killing my host though. This blog is hosted in a shared hosting with a cheap monthly fee. So driving thousands of people coming here would definitely melt-down the server =P
Thanks anyway…
-
Pingback:
Posted: Aug 24th, 2007 at 10:05 pm3SODEVE » Apparently is no Optimization needed for DataTable.Select()[…] losing sleep because of my colleague’s question, I posted the question in .NET Groups where I frequently hang […]
-
xero
Posted: Jan 15th, 2008 at 4:26 am4DataTables are implemented internally with a BTree variation, just like most relational databases use. http://en.wikipedia.org/wiki/B-tree
Searching in from multiple angles and inherent in the structure. Usually, uou can speed things up by adding an index, though I don’t remember if DataTables support indexes. I know that searching on the primary key is incredibly fast compared to not searching on the key. So if you’re not using a PK, start using one.
Another way to speed things up is to process the rows into other tables so the search set is smaller, or to use DataViews to that effect.
-
Posted: Jan 16th, 2008 at 10:05 pm5Thanks for the insight, Xero.
But as for this case, my team ‘decided’ that traversing line-by-line is the best solution since the data is already sorted and we actually need to process each row.
As it turns out, we actually pointed the gun to the wrong suspect. After we run profiler, the bottleneck was not on the DataTable.Select(), but it was caused by invocation of web services.
