JDBC fetch size

make your java run faster

A blog on java performance and optimization. On JDBC, Hibernate, caching, algorithms, profiling and anything that can make your code run faster.

 

 

 

 

 

 

 

Sunday, June 21, 2015

JDBC fetch size - commonly forgotten setting

 

JDBC fetch size parameter can significantly reduce the result set fetching time. It can be set on any Statement or PreparedStatement object. Many frameworks like Spring or Hibernate give you very convenient API to do this.

Although this parameter is widely available, I discovered that a lot of projects I have visited did not configure the fetch size and thus was using its default value. Yet, the default value can result in a poor performance on some jdbc drivers.

I want to show you how much you can improve your fetching performance if you adjust the fetch size to your statements. The scale of improvement depends on the driver you use.

Oracle jdbc driver

Assume we have table with 1 million rows and for any reason we have to fetch all records into JVM. How fast you can get all data? What fetch size will Oracle use if you don’t set it explicitly?

Figure 1 - fetching 1M rows with different fetchSize values (local oracle database)

Figure 1 shows fetching times for different fetch size values for Oracle database. In this example database and java application are located on the same machine. I will show later how it looks for a remote database.

Setting Fetch Size with standard JDBC calls

This is how you can set fetch size for given PreparedStatement using JDBC API:

	PreparedStatement stmt = null;
	ResultSet rs = null;
	try {
	stmt = conn. prepareStatement("select a, b, c from table");
	stmt.setFetchSize(200);
	rs = stmt.executeQuery();
	while (rs.next()) {
	...
	}
	}

view rawAAL1.standardJdbcAPI.java hosted with ❤ by GitHub

Lets see what happens if fetchSize property is set to 10. When rs.next() is called for first time, the oracle driver fetches first 10 records from database and store them in a memory buffer. So, for next 9 calls to rs.next() records are retrieved from this buffer. After the buffer is fully read, subsequent rs.next()will force driver to fetch a new bunch of rows (10) into the buffer.

So if we want to read 10k rows with fetch size set to 10, the driver will make 1000 round trips to the database using the underlying connection. If we set the fetchSize to 500 the driver will perform only 20 round trips to our database.

Look at Figure 1. Setting fetchSize to 100 gives you a 6 times shorter fetching time then with setting fetchSize to 10. Now, you should know that the default fetchSize for the oracle driver is 10...

Two important comments:

• fetchSize can be set on each Statement or PreparedStatement or even on ResultSet. By default, ResultSet uses fetchSize of Statement from which is born. The default value for Statement or PreparedStatementis jdbc driver specific
• fetchSize is only a hint for the driver – the Oracle driver respects this setting, while other drivers may ignore it and fetch all the records at once, for instance.

 

Setting Fetch Size with Spring JdbcTemplate

When using Spring jdbc support you can do this in 2 ways:

Ad hoc JdbcTemplate instance

	JdbcTemplate jdbc = new JdbcTemplate(dataSource);
	jdbc.setFetchSize(200);
	jdbc.query("select a, b, c from table",
	new RowCallbackHandler() {
	@Override
	public void processRow(ResultSet rs) throws SQLException {
	...
	}
	}
	);

view rawAAL2.adhocJdbcTemplate.java hosted with ❤ by GitHub

Shared JdbcTemplate instance

	public class MyJdbcDaoImpl extends JdbcDaoSupport implements MyJdbcDao {
	@Override
	protected void initTemplateConfig() {
	getJdbcTemplate().setFetchSize(200);
	}
	public MyResult loadAll() {
	final MyResult result = new MyResult();
	getJdbcTemplate().query("select a, b, c from table",
	new RowCallbackHandler() {
	@Override
	public void processRow(ResultSet rs) throws SQLException {
	...
	result.add(...);
	}
	}
	);
	} // end of loadAll
	}

view rawAAL3.sharedJdbcTemplate.java hosted with ❤ by GitHub

When implementing a DAO that extends JdbcDaoSupport every call to getJdbcTemplate() returns the same shared JdbcTemplate instance. You can mix this with ad-hoc instances. For example, override initTemplateConfig()to set the default for this DAO but use ad-hoc JdbcTemplate for selected queries.