Spring JDBC Template Tuning

Had a fun little issue the other day with spring's JDBC template. I was querying a fairly large data set, about 65 million records, and realized the job was running slower than molasses going uphill. After debugging the code I found the query was returning very fast, so the database was not the problem. The issue ended up being JDBC template itself, namely the fetch size. If you do not configure the fetch size explicitly the template will use the driver default, which for Oracle is 10, the database I was working with.

The rows returned from my test query was about 53,000, so with a fetch size of 10 this would take  5,300 network trips to retrieve all the data and store it in memory. Before tuning this operation took about 2.5 minutes to run.

One thing to keep in mind when tuning the fetch size is memory consumption. You should be sure you will have enough memory available to store the result set. For my example I set the fetch size to 10,000 as my result sets were always very large. I was also working with a batch job which ran during off-peak hours, so I knew my available memory would be plentiful.


After setting the fetch size the operation took 15 seconds to run, way better than before. Nice!

Sample code:
On a dao class extending JdbcDaoSupport  I stuck this in an overridden initDao() method.
simpleJdbcTemplate.setFetchSize(1000);

Here is another good post about JDBC fetch size: http://webmoli.com/2009/02/01/jdbc-performance-tuning-with-optimal-fetch-size/