A comparison of local caches (2) 【本地缓存之比较 (2)】

接上一篇： A comparison of local caches (1) 【本地缓存之比较 (1)】

This article will compare the asynchronous local caches.

Currently Spring @Cacheable doesn't support async cache by default (we can use some tricks to achieve the goal though).

Guava and Caffiene support async cache with refresh method.

本文会对异步刷新的本地缓存做个介绍。

目前 Spring 的 @Cacheable 注解默认不支持异步刷新（可以使用一些特殊的技巧来实现，这里暂时不提）

Guava 和 Caffiene 的 refresh 方法都对缓存的异步刷新提供了很好的支持。

1. Guava

public class GuavaAsyncCacheExample {
    static AtomicInteger counter = new AtomicInteger(0);

    static ListeningExecutorService exe = MoreExecutors.listeningDecorator(Executors.newWorkStealingPool());

    static LoadingCache<Integer, Integer> refreshCache = CacheBuilder.newBuilder()
            .refreshAfterWrite(3, TimeUnit.SECONDS)
            .recordStats()
            .build(new CacheLoader<Integer, Integer>() {
                @Override
                public Integer load(Integer key) throws Exception {
                    return getIntegerCache();
                }

                public ListenableFuture<Integer> reload(Integer key, Integer oldValue) throws Exception {
                    return exe.submit(() -> getIntegerCache());
                }
            });

    private static Integer getIntegerCache() throws Exception {
        System.out.println("populate cache " + LocalDateTime.now());
        Thread.sleep(2000);
        //if (counter.incrementAndGet() >= 2) throw new Exception("error");
        return counter.incrementAndGet() * 10;
    }

    public static void main(String[] args) throws Exception {
        useAsyncCache();
    }

    static void useAsyncCache() throws Exception {
        for (int i = 0; i < 10; i++) {
            System.out.println(refreshCache.get(0) + " -- " + LocalDateTime.now());
            Thread.sleep(1000);
        }
    }
}

Console output:

populate cache 2017-06-12T15:21:11.026
10 -- 2017-06-12T15:21:13.028
10 -- 2017-06-12T15:21:14.029
10 -- 2017-06-12T15:21:15.029
populate cache 2017-06-12T15:21:16.046
10 -- 2017-06-12T15:21:16.068
10 -- 2017-06-12T15:21:17.069
20 -- 2017-06-12T15:21:18.069
20 -- 2017-06-12T15:21:19.069
20 -- 2017-06-12T15:21:20.069
20 -- 2017-06-12T15:21:21.069
populate cache 2017-06-12T15:21:21.069
20 -- 2017-06-12T15:21:22.070

Be very careful that if we omit the reload method when implementing CacheLoader, the cache will not be load in async way anymore.

注意下，如果不小心忘了重写 reload 方法，缓存将不再被异步刷新。

//                public ListenableFuture<Integer> reload(Integer key, Integer oldValue) throws Exception {
//                    return exe.submit(() -> getIntegerCache());
//                }

populate cache 2017-06-12T15:30:58.924
10 -- 2017-06-12T15:31:00.928
10 -- 2017-06-12T15:31:01.928
10 -- 2017-06-12T15:31:02.928
populate cache 2017-06-12T15:31:03.929
20 -- 2017-06-12T15:31:05.960
20 -- 2017-06-12T15:31:06.960
20 -- 2017-06-12T15:31:07.960
populate cache 2017-06-12T15:31:08.961
30 -- 2017-06-12T15:31:10.961
30 -- 2017-06-12T15:31:11.962
30 -- 2017-06-12T15:31:12.962
populate cache 2017-06-12T15:31:13.962
40 -- 2017-06-12T15:31:15.963

Previously cache can be retrieved immediately. After removing the reload method, the get method will be blocked until load method completes.

上图的日志就是在注释掉 reload 方法的情况下得到的。可以明显的看到当缓存过期之后，再次取缓存会被缓慢的 load 方法阻塞掉。

Hmmm, what happens if an error occurs during cache loading? Let's modify the loading part a little bit.

对了，如果 load 方法执行过程中不小心出错会怎么样呢? 我们来稍微修改下加载逻辑。

    private static Integer getIntegerCache() throws Exception {
        System.out.println("populate cache " + LocalDateTime.now());
        Thread.sleep(2000);
        if (counter.incrementAndGet() >= 2) throw new Exception("error");
        return counter.get() * 10;
    }

Accordign to the output, guava gives warning message, but the old value is still returned.

通过输出日志可以看到，Guava会抛出警告日志，返回旧值。

populate cache 2017-06-12T15:37:08.437
10 -- 2017-06-12T15:37:10.440
10 -- 2017-06-12T15:37:11.441
10 -- 2017-06-12T15:37:12.441
populate cache 2017-06-12T15:37:13.441
Jun 12, 2017 3:37:15 PM com.google.common.cache.LocalCache$Segment$1 run
WARNING: Exception thrown during refresh
java.util.concurrent.ExecutionException: java.lang.Exception: error
    at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
    at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
    at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
    at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:135)
    at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2346)
    at com.google.common.cache.LocalCache$Segment$1.run(LocalCache.java:2329)
    at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
    at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
    at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:101)
    at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:170)
    at com.google.common.cache.LocalCache$Segment.loadAsync(LocalCache.java:2324)
    at com.google.common.cache.LocalCache$Segment.refresh(LocalCache.java:2387)
    at com.google.common.cache.LocalCache$Segment.scheduleRefresh(LocalCache.java:2365)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2185)
    at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
    at cachecmp.GuavaAsyncCacheExample.useAsyncCache(GuavaAsyncCacheExample.java:49)
    at cachecmp.GuavaAsyncCacheExample.main(GuavaAsyncCacheExample.java:44)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: java.lang.Exception: error
    at cachecmp.GuavaAsyncCacheExample.getIntegerCache(GuavaAsyncCacheExample.java:39)
    at cachecmp.GuavaAsyncCacheExample.access$000(GuavaAsyncCacheExample.java:17)
    at cachecmp.GuavaAsyncCacheExample$1.load(GuavaAsyncCacheExample.java:28)
    at cachecmp.GuavaAsyncCacheExample$1.load(GuavaAsyncCacheExample.java:25)
    at com.google.common.cache.CacheLoader.reload(CacheLoader.java:97)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
    at com.google.common.cache.LocalCache$Segment.loadAsync(LocalCache.java:2323)
    ... 13 more

10 -- 2017-06-12T15:37:15.475
populate cache 2017-06-12T15:37:16.476
Jun 12, 2017 3:37:18 PM com.google.common.cache.LocalCache$Segment$1 run
10 -- 2017-06-12T15:37:18.478

2. Caffiene

public class CaffeineAsyncCacheExample {
    static AtomicInteger counter = new AtomicInteger(0);

    static LoadingCache<Integer, Integer> refreshCache = Caffeine.newBuilder()
            .refreshAfterWrite(3, TimeUnit.SECONDS)
            .recordStats()
            .build(new CacheLoader<Integer, Integer>() {
                @Override
                public Integer load(Integer key) throws Exception {
                    return getIntegerCache();
                }
            });

    private static Integer getIntegerCache() throws Exception {
        System.out.println("populate cache " + LocalDateTime.now());
        Thread.sleep(2000);
        return counter.incrementAndGet() * 10;
    }

    public static void main(String[] args) throws Exception {
        useAsyncCache();
    }

    static void useAsyncCache() throws Exception {
        for (int i = 0; i < 10; i++) {
            System.out.println(refreshCache.get(0) + " -- " + LocalDateTime.now());
            Thread.sleep(1000);
        }
    }
}

Opps, again, the reload method is omitted. But the output looks perfectly fine, what happened?

啊哦，一不小心又忘记写 reload 方法了。 然而仔细一看输出日志，竟毫无阻塞的问题。

populate cache 2017-06-12T15:49:48.043
10 -- 2017-06-12T15:49:50.047
10 -- 2017-06-12T15:49:51.047
10 -- 2017-06-12T15:49:52.047
populate cache 2017-06-12T15:49:53.063
10 -- 2017-06-12T15:49:53.086
10 -- 2017-06-12T15:49:54.087
20 -- 2017-06-12T15:49:55.088
20 -- 2017-06-12T15:49:56.088
20 -- 2017-06-12T15:49:57.089
20 -- 2017-06-12T15:49:58.089
populate cache 2017-06-12T15:49:58.089
20 -- 2017-06-12T15:49:59.090

In order to dig out the truth, let's take a look at the source code of CacheLoader.

为了探索背后的原因，我们来看看CacheLoader的源码

Guava

public abstract class CacheLoader<K, V> {
    protected CacheLoader() {
    }

    public abstract V load(K var1) throws Exception;

    @GwtIncompatible("Futures")
    public ListenableFuture<V> reload(K key, V oldValue) throws Exception {
        Preconditions.checkNotNull(key);
        Preconditions.checkNotNull(oldValue);
        return Futures.immediateFuture(this.load(key));
    }

......

Caffiene

@FunctionalInterface
@ThreadSafe
public interface CacheLoader<K, V> extends AsyncCacheLoader<K, V> {
    @CheckForNull
    V load(@Nonnull K var1) throws Exception;

    @CheckForNull
    default V reload(@Nonnull K key, @Nonnull V oldValue) throws Exception {
        return this.load(key);
    }

    @Nonnull
    default CompletableFuture<V> asyncReload(@Nonnull K key, @Nonnull V oldValue, @Nonnull Executor executor) {
        Objects.requireNonNull(key);
        Objects.requireNonNull(executor);
        return CompletableFuture.supplyAsync(() -> {
            try {
                return this.reload(key, oldValue);
            } catch (RuntimeException var4) {
                throw var4;
            } catch (Exception var5) {
                throw new CompletionException(var5);
            }
        }, executor);
    }

...

By default, Guava calls reload method while Caffiene calls asyncReload method, that's why we only need to rewrite load method in Caffiene in stead. It saves us some coding effort, but more importantly, it decreases the possibility to make mistakes. For example, some one may forgot to update the reload method after making some changes in load block.

默认情况下，Guava 调用 reload 方法，reload 又会去阻塞调用 load 方法， 而Caffiene 默认调用 asyncReload 方法，并不阻塞，这就是为什么使用 Caffiene 的时候只需要重写 load 方法即可。这不光是节省了一点代码，更重要的，它降低了之后出错的几率。很有可能某位别的程序员日后修改了 load 方法的实现 却忘记了 reload 方法，于是出现错误。