SQL Fetch size

JDBC performance tuning with optimal fetch size

February 1, 2009 31 Comments

Tuning performance using fetch size is an old technique some of you might already be using this configuration; some may know about it but may not have implemented. Recently I have implemented in my current project, would like to share my experience. In a typical production environment database and application will be running on different physical server. Even if you have high-end server class machine the network traffic between application and database server is one of the key factor of your application performance.

Most of the JDBC drivers’ default fetch size is 10. In normal JDBC programming if you want to retrieve 1000 rows it requires 100 network round trips between your application and database server to transfer all data. Definitely this will impact your application response time. The reason is JDBC drivers are designed to fetch small number of rows from database to avoid any out of memory issues. For example if your query retrieves 1 million rows, the JVM heap memory may not be good enough to hold that large amount of data hence JDBC drivers are designed to retrieve small number (10 rows) of rows at a time that way it can support any number of rows as long as you have better design to handle large row set at your application coding. If you configure fetch size as 100, number of network trips to database will become 10. This will dramatically improve performance of your application.

MySQL

When I was working on MySQL I felt its performance is always better than SQL Server and Oracle. The reason is by default, ResultSets are completely retrieved from database server.

Reference: http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-implementation-notes.html

Oracle

The default fetch size is 10. So you tend to find Oracle retrieval performance is slower than other server. You can observe executing a query in SQL Plus is faster than JDBC. You can change default fetch size by setting connection property “defaultRowPrefetch”.

Reference: http://download-uk.oracle.com/docs/cd/B25221_04/web.1013/b13593/optimiz011.htm#BEEBHBBG

SQL Server

By default it retrieves all the rows from database unless you specify cursor type in the JDBC driver. Like MySQL you can observe SQL Server JDBC performance is better than Oracle.

Reference: http://technet.microsoft.com/en-us/library/aa342344(SQL.90).aspx

DB2

Seems the default fetch size is 32. Don’t know much about this database as I don’t have this database server. You can change default fetch size by through connection property “block size”.

Important note to consider when tuning fetch size:

1. Make sure your JDBC driver supports configuring fetch size.
2. The fetch size should be based on your JVM heap memory setting. Since JVM heap size varies between different environment, don’t hard-code fetch size keep it as configurable parameters.
3. If your fetch size is large, you might encounter out of memory issue. For example a less number of column tables might support large rows fetch size than more number of columns.
4. You can set fetch size based certain datatype like blob, image, etc. We follow certain naming convention for columns for example all image and blob column will have suffix “Blob”. I set high fetch size if the query doesn’t contain “Blob” word otherwise set low fetch size.

Sample Code:

Connection connection = DriverManager.getConnection("");
Statement statement = connection.createStatement();
statement.setFetchSize(1000); // configure the fetch size
ResultSet resultSet = statement.executeQuery("");

http://webmoli.com/2009/02/01/jdbc-performance-tuning-with-optimal-fetch-size/