用了这么久的Log4j这次倒下了,而且官方也还没有给出解决方案。

描述:tomcat 经过一天多时间的访问,出现了hang ,使用 Jstack 查看堆栈后,发现现成 blocked ,主要是 Log4j 的ROOTlock了。

"http-nio-2180-exec-9" #35 daemon prio=5 os_prio=0 tid=0x0000000021a8b000 nid=0x12a8 runnable [0x000000002e5ee000]
java.lang.Thread.State: RUNNABLE
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(Unknown Source)
at java.io.BufferedOutputStream.write(Unknown Source)
- locked <0x00000006c0cf7380> (a java.io.BufferedOutputStream)
at java.io.PrintStream.write(Unknown Source)
- locked <0x00000006c0cf7360> (a java.io.PrintStream)
at org.apache.tomcat.util.log.SystemLogHandler.write(SystemLogHandler.java:172)
at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source)
at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source)
at sun.nio.cs.StreamEncoder.implFlush(Unknown Source)
at sun.nio.cs.StreamEncoder.flush(Unknown Source)
- locked <0x00000006c0eb2428> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flush(Unknown Source)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
- locked <0x00000006c0eaf3b8> (a org.apache.log4j.ConsoleAppender)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
- locked <0x00000006c0ea85c8> (a org.apache.log4j.spi.RootLogger)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:155)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:861)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:857)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:617)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1527)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1484)
- locked <0x00000006c27752a8> (a org.apache.tomcat.util.net.NioChannel)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)

被锁住的线程堆栈:

"http-nio-2180-exec-10" #36 daemon prio=5 os_prio=0 tid=0x0000000021a8c000 nid=0xcfc waiting for monitor entry [0x000000002e49e000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.log4j.Category.callAppenders(Category.java:204)
- waiting to lock <0x00000006c0ea85c8> (a org.apache.log4j.spi.RootLogger)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.apache.commons.logging.impl.Log4JLogger.debug(Log4JLogger.java:155)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:861)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:857)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:617)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1527)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1484)
- locked <0x00000006c2723a10> (a org.apache.tomcat.util.net.NioChannel)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)

查了很多资料,比较全的是这个老兄回答的:

http://stackoverflow.com/questions/3537870/production-settings-file-for-log4j/

第一个回答的截取在这里,备份下,解决方案他也提到了,不过只是减缓,没有完全解决。

Log4j 1.2 is vulnerable to deadlocks when toString() produces nested logging.

See old aged still unresolved issues like Log4J can create deadlock conditions (concurrent package donation) and Deadlock with RollingFileAppender.

It also has performance-killing lock synchronization issues under heavy concurrent load. LikeCategory callAppenders synchronization causes java.lang.Thread.State: BLOCKED and Move org.apache.log4j.Category to reentrant read/write locks.

Even AsyncAppender is not free of excessive locks: AsyncAppender.doAppend() does not need to be synchronized and Deadlock in 1.2.15 caused by AsyncAppender and ThrowableInformation classes. Also beware of AsyncAppender overflow.

One caveat is to always constrain root category level to at least INFO or higher. This would prevent logging calls from acquiring unnecessary locks mentioned in the above issues. Just limiting appender threshold is not sufficient as it is taken into account later. See explanation with publish/subscribe analogy. Nested categories of interest can be individually given lower priorities if needed.

log4j deadlock的更多相关文章

  1. 跨过slf4j和logback,直接晋级log4j 2

    今年一直关注log4j 2,但至今还没有出正式版.等不及了,今天正式向大家介绍一下log4j的升级框架,log4j 2. log4j,相信大家都熟悉,至今对java影响最大的logging系统,至今仍 ...

  2. 一个与Log4j相关的死锁(转)

    这个死锁的原因:一个动作需要两个临界对象.静态同步方法,就是一个临界对象.这种场景,静态同步方法每次只能有一个线程持有.如果存在另一个临界对象,静态同步方法中也需要获取这个临界对象.即一个动作需要两个 ...

  3. 利用log4j+mongodb实现分布式系统中日志统一管理

    背景     在分布式系统当中,我们有各种各样的WebService,这些服务可能分别部署在不同的服务器上,并且有各自的日志输出.为了方便对这些日志进行统一管理和分析.我们可以将日志统一输出到指定的数 ...

  4. org.apache.log4j.Logger详解

    org.apache.log4j.Logger 详解 1. 概述 1.1. 背景 在应用程序中添加日志记录总的来说基于三个目的 :监视代码中变量的变化情况,周期性的记录到文件中供其他应用进行统计分析工 ...

  5. (转)配置Log4j(很详细)

    来自:http://blog.csdn.net/yttcjj/article/details/37957317 Log4J的配置文件(Configuration File)就是用来设置记录器的级别.存 ...

  6. Log4j记录日志步骤

    记录日志对调试Bug很有帮助  亲身体会 个人习惯用Log4J,大家可以在apache网站:jakarta.apache.org/log4j 可以免费下载到Log4j最新版本的软件包. Log4j支持 ...

  7. log4j:ERROR Failed to rename [/log/xxx.log] to [/log/xxx.log.2016-11-23.log]

    Log4j报错: log4j:ERROR Failed to rename [/log/xxx.log] to [/log/xxx.log.2016-11-23.log] google了一下发现是个b ...

  8. 项目自动化建构工具gradle 入门2——log4j输出helloWorld

    上一章节呢,有一个能跑的程序了.但是对做工程的人来说,用日志输出感觉比用System.out要有档次一点.比如使用log4j.直接上例子: 1进入D:\work\gradle\log目录  ,您电脑没 ...

  9. java日志组件介绍(common-logging,log4j,slf4j,logback )

    转自:http://www.blogjava.net/daiyongzhi/archive/2014/04/13/412364.html common-logging是apache提供的一个通用的日志 ...

随机推荐

  1. poj1273

    赤裸裸的最大流 #include <iostream> #include <cstdio> #include <cstdlib> #include <cstr ...

  2. python基础--time和datetime模块

    一:说明在Python中,通常有这几种方式来表示时间:1)时间戳 2)格式化的时间字符串 3)元组(struct_time)共九个元素.由于Python的time模块实现主要调用C库,所以各个平台可能 ...

  3. django startproject xxx:报错UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 13: ordinal not in range(128)

    django startproject xxx:报错UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 13: o ...

  4. Hitcon 2016 Pwn赛题学习

    PS:这是我很久以前写的,大概是去年刚结束Hitcon2016时写的.写完之后就丢在硬盘里没管了,最近翻出来才想起来写过这个,索性发出来 0x0 前言 Hitcon个人感觉是高质量的比赛,相比国内的C ...

  5. 启动DELPHI2010出现 EditorLineEnds.ttr 错误的解决方法

      在网上找到了很多方法.其实解决这个问题的方法, 最简单的就是把  EditorLineEnds.ttr  改名为  EditorLineEnds.ttf 然后,安装它, 安装完成后就OK了.

  6. 【Ubuntu】Ubuntu设置和查看环境变量

    [Ubuntu]Ubuntu设置和查看环境变量    转载 https://blog.csdn.net/White_Idiot/article/details/78253004 1. 查看环境变量 查 ...

  7. Vue父子组件之间通信

    1.父 -> 子.通过props //father.vue <template> <div id="father"> <div><l ...

  8. AI技术在智能海报设计中的应用

    背景 在视觉设计领域中,设计师们往往会因为一些简单需求付出相当多的时间,比如修改文案内容,设计简单的海报版式,针对不同机型.展位的多尺寸拓展等.这些工作需要耗费大量的时间.人力成本(5~6张/人日), ...

  9. CF815C Karen and Supermarket

    题目链接 CF815C Karen and Supermarket 题解 只要在最大化数量的前提下,最小化花费就好了 这个数量枚举ok, dp[i][j][1/0]表示节点i的子树中买了j件商品 i ...

  10. 1063 合并果子 2004年NOIP全国联赛普及组

    题目描述 Description 在一个果园里,多多已经将所有的果子打了下来,而且按果子的不同种类分成了不同的堆.多多决定把所有的果子合成一堆. 每一次合并,多多可以把两堆果子合并到一起,消耗的体力等 ...