Reloading Java Classes 201: How do ClassLoader leaks happen? Translation

The original link : http://zeroturnaround.com/rebellabs/rjc201/

From ClassLoaders to Classes

从ClassLoader到Classes

If you have programmed in Java for some time you know that memory leaks do happen. Usually it’s the case of a collection somewhere with references to objects (e.g. listeners) that should have been cleared, but never were. Classloaders are a very special case of this, and unfortunately, with the current state of the Java platform, these leaks are both inevitable and costly: routinely causing OutOfMemoryError’s in production applications after just a few redeploys.

如果你已经使用java编程一段时间了, 你知道内存泄露在java中时有发生.常见的例子是集合中的对象(比如listeners)的引用该清理掉却没从没被清理.Classloader是这种情况的非常特殊的一个案例,而且非常不幸的是,Java平台的现状, 这些泄露不可避免而且造成的代价很高: 通常在几次redeploy以后再引用程序中就会造成OutOfMemoryError.

Let’s get started. Recalling RJC101: to reload a class we threw away the old classloader and created a new one, copying the object graph as best we could:

让我们开始吧. 回顾 RJC101: reload一个class我们丢弃了旧的classloader并且创建了一个新的, 然后尽力复制原来的对象图:

Every object had a reference to its class, which in turn had a reference to its classloader. However we didn’t mention that every classloader in turn has a reference to each of the classes it has loaded, each of which holds static fields defined in the class:

每一个object都有一个指向它的class的引用, 同时它的class也有一个指向它的classloader的引用.然而我们并没有提到每个classloader反过来也对每个载入的class持有一个引用, 每个class都有一个static的field

This means that

这意味着

1. If a classloader is leaked it will hold on to all its classes and all their static fields. Static fields commonly hold caches, singleton objects, and various configuration and application states. Even if your application doesn’t have any large static caches, it doesn’t mean that the framework you use doesn’t hold them for you (e.g. Log4J is a common culprit as it’s often put in the server classpath). This explains why leaking a classloader can be so expensive.
2. To leak a classloader it’s enough to leave a reference to any object, created from a class, loaded by that classloader. Even if that object seems completely harmless (e.g. doesn’t have a single field), it will still hold on to its classloader and all the application state. A single place in the application that survives the redeploy and doesn’t do a proper cleanup is enough to sprout the leak. In a typical application there will be several such places, some of them almost impossible to fix due to the way third-party libraries are built. Leaking a classloader is therefore, quite common.

1. 假如一个classloader泄露了, 他将会持有所有他的class和他们的static field.Static field通常是cache, 单例 object, 和变化的配置及程序状态变量.即使你的程序没有任何打的静态cache, 也不代表你用的框架没有(比如Log4J就是一个常见的罪魁祸首, 因为它经常被load进server的classpath中). 这解释了为什么泄露一个classloader会造成很严重的问题
2. 泄露一个classloader会让任何对象被这个classloader加载的class创建的object的引用孤立. 即使这个object看起来是完全无害的(这个object没有任何一个单独的filed),他依然会被他的classloader和所有application state持有.一块在redepoly中存活下来的application中的独立的空间如果没有做一个合适的清理,就足以造成这种泄露.一个典型的application会有好几个这样的空间, 有一些由于第三方库的构建方法,甚至是不可能修复的. 所以说classloader的泄露太常见了.

To examine this from a different perspective let’s return to the code example from our previous article. Breeze through it to quickly catch up.

为了用另一种视角来解释这种现象,我们回到上一个章节提到的代码.快速的过一下:

Introducing the Leak

泄露介绍

We will use the exact same Main class as before to show what a simple leak could look like:

我们会使用一个和前面完全一样的Main class来演示一个简单的泄露

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

public class Main { private static IExample example1; private static IExample example2;   public static void main(String[] args) { example1 = ExampleFactory.newInstance().copy();   while (true) { example2 = ExampleFactory.newInstance().copy();   System.out.println("1) " + example1.message() + " = " + example1.plusPlus()); System.out.println("2) " + example2.message() + " = " + example2.plusPlus()); System.out.println();   Thread.currentThread().sleep(3000); } } }

The ExampleFactory class is also exactly the same, but here’s where things get leaky. Let’s introduce a new class called Leak and a corresponding interface ILeak:

ExampleFactory class也是完全一样的, 但这里就是泄露发生的地方. 我们介绍一下Leak和他对应的接口ILeak

1 2 3 4 5 6 7 8 9 10

interface ILeak { }   public class Leak implements ILeak { private ILeak leak;   public Leak(ILeak leak) { this.leak = leak; } }

As you can see it’s not a terribly complicated class: it just forms a chain of objects, with each doing nothing more than holding a reference to the previous one. We will modify the Example class to include a reference to the Leak object and throw in a large array to take up memory (it represents a large cache). Let’s omit some methods shown in the previous article for brevity:

如你所见这是一个简单的class: 他只是构造了一个对象链, 除了持有一个对象引用啥也没干.我们修改Example类将Leak包含进来, 然后创建一个大数组中来占据内存空间(他代表了一个大cache). 为了简单起见我们忽略了上一篇文章提到的一些方法:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

public class Example implements IExample { private int counter; private ILeak leak;   private static final long[] cache = new long[1000000];   /* message(), counter(), plusPlus() impls */   public ILeak leak() { return new Leak(leak); }   public IExample copy(IExample example) { if (example != null) { counter = example.counter(); leak = example.leak(); } return this; } }

The important things to note about Example class are:

Example类中最重要的是

1. Example holds a reference to Leak, but Leak has no references to Example.
2. When Example is copied (method copy() is called) a new Leak object is created holding a reference to the previous one.

1. Example持有一个Leak的引用,但是Leak中没有对Example的引用
2. 当Example被copy(copy()方法被调用),一个新的Leak对象就会被创建并持有前一个Leak的引用

If you try to run this code an OutOfMemoryError will be thrown after just a few iterations:

尝试运行这段代码过不久就会造成OutOfMemoryError

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at example.Example.<clinit>(Example.java:8)

With the right tools, we can look deeper and see how this happens.

使用正确的工具, 我们可以更深入的看到OOM是怎么发生的

Post Mortem

Since Java 5.0, we’ve been able to use the jmap command line tool included in the JDK distribution to dump the heap of a running application (or for that matter even extract the Java heap from a core dump). However, since our application is crashing we will need a feature that was introduced in Java 6.0: dumping the heap on OutOfMemoryError. To do that we only need to add -XX:+HeapDumpOnOutOfMemoryError to the JVM command line:

从Java 5.0开始,我们可以使用JDK中的jmap命令行工具来导出运行程序的heap(对于这个问题甚至可以将整个Java的heap从core中导出).然而, 因为我们的程序crash了, 我们需要一个Java6.0中的特性: 在OutOfMemoryError时导出heap.为了做到这点我们仅仅需要加入命令 -XX:+heapDumpOnOutOfMemoryError 到JVM命令行中:

java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid37266.hprof ...
Heap dump file created [57715044 bytes in 1.707 secs]
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at example.Example.<clinit>(Example.java:8)

After we have the heap dump we can analyze it. There are a number of tools (including jhat, a small web-based analyzer included with the JDK), but here we will use the more sophisticatedEclipse Memory Analyzer (EMA).

导出heap后我们可以开始分析了. 这里有好几种工具可以使用(包括 jhat, JDK里一个小巧的基于web的分析器), 但是这里我们会使用精密的Eclipse Memory Analyzer(EMA)

After loading the heap dump into the EMA we can look at the Dominator Tree analysis. It is a very useful analysis that will usually reliably identify the biggest memory consumers in the heap and what objects hold a reference to them. In our case it seems quite obvious that the Leak class is the one that consumes most of the heap:

通过载入heap dump到EMA, 我们可以看看Dominator Tree的分析结果. 这是非常有用的分析, 他可靠的辨认出了heap里最大的内存消耗者和什么对象持有这些消耗者的引用.在我们的例子里很明显Leak就是最大的消耗者:

Now let’s run a search for all of the Leak objects and see what are they holding to. To do that we run a search List objects -> with outgoing references for “example.Leak”:

现在让我们搜索一下所有的Leak对象并看看他们都持有什么.按下面步骤运行 对example.Leak执行 List Objects -> with outgoing references

The results include several Leak objects. Expanding the outgoing references we can see that each of them holds on to a separate instance of Example through a bunch of intermediate objects:

这些结果包括一些Leak对象. 展开这些引用,我们可以看到,他们每个都通过一群中间对象持有一个Example对象实例

You may notice that one of the intermediate objects is ExampleFactory$1, which refers to the anonymous subclass of URLClassLoader we created in the ExampleFactory class. In fact what is happening is exactly the situation we described in the beginning of the article:

你可能注意到了其中一个中间对象是ExampleFactory$1, 他指向的是我们在ExampleFactory类里创建的URLClassLoader的一个匿名子类. 事实上发生的事情正是我们在文章开头讨论的情况

• Each Leak object is leaking. They are holding on to their classloaders
• 每个Leak都在泄露.他们持有他们的classloader
• The classloaders are holding onto the Example class they have loaded:
• classloader持有所有他们已经load的class

Conclusions

结论

Though this example is slightly contrived, the main idea to take away is that it’s easy to leak a single object in Java. Each leak has the potential to leak the whole classloader if the application is redeployed or otherwise a new classloader is created. Since preventing such leaks is very challenging, it’s a better idea to use Eclipse Memory Analyzer and your understanding of classloaders to hunt them down after you get an OutOfMemoryError on redeploy.

虽然这个例子有点人为而之, 但他的中心思想是在Java中泄露一个object. 假如程序被redeployed或者创建新的classloader,那么每次泄露都有潜在的泄露整个classloader的危险. 因为防止这样的泄露是很难的, 所以最好的主义是在你redeploy以后获得OutOfMemoryError是使用EMA和你对classloader的理解来解决这个问题.

This article addressed the following questions:

这篇文章提出了以下问题

• How does reloading a class cause the classloader to leak?
• 如何重加载一个class会导致classloader泄露
• What are some consequences of leaking classloaders?
• 泄露classloader的后果是什么
• What tools can be used to troubleshoot these memory leaks?
• 什么工具可以用来排除这些内存泄露故障

Resources

• How to fix the dreaded “java.lang.OutOfMemoryError: PermGen space” exception (classloader leaks)
• Download full source code