从容器里dump java堆实验探索(原创)
目标:从docker容器里dump java堆
模拟程序
占用空间500M,
设置启动JVM参数
docker启动命令
(PS:经过测试,至少要650M才能启动容器)
方式1: 通过docker exec
先通过 docker exec $containerid ps x 获取进程号
执行 docker exec
此命令实际就是进入docker容器里执行/jdk/bin/jmap dump ,导出的文件也是存放在容器里
经过多次实验,基本上每次都触发容器killed
方式2,: 通过nsenter
参考文章 https://github.com/jpetazzo/nsenter
这里有句重点是”evades resource limitations” ,躲避资源限制
我们使用nsenter试验一下
使用docker-enter (docker-enter脚本 调用了nsenter)
多次测试,都没有发生docker killed现象
结论: 通过nsenter 方式去 dump java 堆,能比较高成功率, 但仅限于此实验场景,不代表其他线上场景一定能成功
PS: jdk10 新特性里虽然增加了更好的docker支持,但没有提及过jmap 有更好的dump方式
https://bugs.openjdk.java.net/browse/JDK-8146115
To correct these shortcomings and make this support more robust, here's a list of the current cgroup subsystems that we be examined in order to update the internal VM and core library configuration.
Number of CPUs
-----------------------
Use a combination of number_of_cpus() and
cpu_sets() in order to determine how many processors are available to the
process and adjust the JVMs os::active_processor_count appropriately. The
number_of_cpus() will be calculated based on the cpu_quota() and cpu_period()
using this formula: number_of_cpus() = cpu_quota() / cpu_period(). If
cpu_shares has been setup for the container, the number_of_cpus() will be
calculated based on cpu_shares()/1024. 1024 is the default and standard unit
for calculating relative cpu usage in cloud based container management
software.
Also add a new VM flag
(-XX:ActiveProcessorCount=xx) that allows the number of CPUs to be overridden.
This flag will be honored even if UseContainerSupport is not enabled.
Total available memory
-------------------------------
Use the memory_limit() value from the cgroup
file system to initialize the os::physical_memory() value in the VM. This value
will propagate to all other parts of the Java runtime.
Memory usage
--------------------
Use memory_usage_in_bytes() for providing
os::available_memory() by subtracting the usage from the total available memory
allocated to the container.
As as troubleshooting aid, we will dump any
available container statistics to the hotspot error log and add container
specific information to the JVM logging system. Unified Logging will be added
to help to diagnose issue related to this support. Use -Xlog:os+container=trace
for maximum logging of container information.
A new option -XX:-UseContainerSupport will be
added to allow the container support to be disabled. The default for this flag
will be true. Container support will be enabled by default.
PS: jmap –F 那点事
When run without -F
these
tools use Dynamic Attach Mechanism. This
works as follows.
- Before connecting to Java process 1234,
jmap
creates
a file.attach_pid1234
at the working directory of the target process or
at/tmp
. - Then
jmap
sendsSIGQUIT
to the target process. When JVM catches the signal
and finds.attach_pid1234
, it startsAttachListener
thread. AttachListener
thread creates UNIX domain socket/tmp/.java_pid1234
to listen to commands from external tools.- For security reasons when a connection (from
jmap
) is
accepted, JVM verifies that credentials of the socket peer are equal toeuid
andegid
of
JVM process. That's whyjmap
will not work if run by different user (even by
root). jmap
connects
to the socket, and sendsdumpheap
command.- This command is read and executed by
AttachListener
thread of the JVM. All output is sent back to the
socket. Since the heap dump is made in-process directly by JVM, the operation
is really fast. However, JVM can do this only at safepoints. If a safepoint cannot be reached (e.g. the
process is hung, not responding, or a long GC is in progress),jmap
will
timeout and fail.
Let's
summarize the benefits and the drawbacks of Dynamic Attach.
Pros.
- Heap
dump and other operations are run collaboratively by JVM at the maximum speed. - You can
use any version ofjmap
orjstack
to connect to any other version of JVM.
Cons.
- The
tool should be run by the same user (euid
/egid
) as the target JVM. - Can be
used only on live and healthy JVM. - Will
not work if the target JVM is started with-XX:+DisableAttachMechanism
.
jmap -F / jstack -F
When run with -F
the
tools switch to special mode that features HotSpot Serviceability Agent. In
this mode the target process is frozen; the tools read its memory via OS
debugging facilities, namely, ptrace
on
Linux.
jmap -F
invokesPTRACE_ATTACH
on the target JVM. The target process is
unconditionally suspended in response toSIGSTOP
signal.- The tool reads JVM memory using
PTRACE_PEEKDATA
.ptrace
can read only one word at a time, so too many calls
required to read the large heap of the target process. This is very and very
slow. - The tool reconstructs JVM internal structures based on the
knowledge of the particular JVM version. Since different versions of JVM have
different memory layout,-F
mode works only ifjmap
comes
from the same JDK as the target Java process. - The tool creates heap dump itself and then resumes the
target process.
Pros.
- No
cooperation from target JVM is required. Can be used even on a hung process. ptrace
works whenever OS-level privileges are enough.
E.g.root
can dump processes of all other users.
Cons.
- Very
slow for large heaps. - The
tool and the target process should be from the same version of JDK.
The safepoint is not guaranteed when the tool attaches in forced mode.
Though jmap
tries to handle
all special cases, sometimes it may happen that target JVM is not in a
consistent state.
方式3: 直接在宿主机执行jmap
首先要先获取到容器里映射到宿主极的进程号
21025就是映射到宿主机的进程id
调用jmap –F 21025 ,但是失败了
没有任何头绪之际,突然发现
exe -> /jdk/bin/java 是红色, 这是容器里的java路径, 但宿主机没有这个路径, 会不会是这里有影响, 赶紧在宿主机也创建这个路径
再次执行jmap, 成功导出
从容器里dump java堆实验探索(原创)的更多相关文章
- 容器中的Java堆大小调整:快速,轻松
在上一篇博客中,我们已经看到Java进行了改进,可以根据正在运行的环境(即物理机或容器(码头工人))识别内存.java的最初问题是,它无法弄清楚它是否在容器中运行,并且它曾经为容器运行所在的整个硬件捕 ...
- 如何规避容器内做Java堆dump导致容器崩溃的问题
写在前边 最近公司生产环境的容器云上出了个性能问题,为了做性能分析,使用 JDK 自带的 jmap 收集堆dump,出现了内存溢出导致了容器崩溃. 本篇文章将带你探究,如何规避容器内做堆 dump 导 ...
- 在 Docker 里跑 Java,你必须知道的那些事儿!(转)
原文 https://www.jianshu.com/p/0897d0581872 背景:众所周知,当我们执行没有任何调优参数(如“java-jar mypplication-fat.jar”)的 J ...
- Java内存泄漏分析系列之六:JVM Heap Dump(堆转储文件)的生成和MAT的使用
原文地址:http://www.javatang.com JVM Heap Dump(堆转储文件)的生成 正如Thread Dump文件记录了当时JVM中线程运行的情况一样,Heap Dump记录了J ...
- 从Java进程里dump出类的字节码文件
想要查看一些被增强过的类的字节码,或者一些AOP框架的生成类,就需要dump出运行时的Java进程里的字节码. 从运行的java进程里dump出运行中的类的class文件的方法: 用agent att ...
- MAT工具定位分析Java堆内存泄漏问题方法
一.MAT概述与安装 MAT,全称Memory Analysis Tools,是一款分析Java堆内存的工具,可以快速定位到堆内泄漏问题.该工具提供了两种使用方式,一种是插件版,可以安装到Eclips ...
- 如何在宿主机上执行容器里的jmap,jtack,jstat 命令获取信息(原创)
一般情况下,我们要获取docker容器里的jvm信息只能进入容器后执行jmap,jstack,jstat 命令去获取,jstack,jstat还好,但是jmap dump的文件要拿出来,得先copy ...
- Java堆内存设置
转自:https://blog.csdn.net/Qiuzhongweiwei/article/details/81023645 堆内存设置 原理 JVM堆内存分为2块:永久空间和堆空间. 永久即持久 ...
- Java堆内存又溢出了!教你一招必杀技
JAVA堆内存管理是影响性能主要因素之一.堆内存溢出是JAVA项目非常常见的故障,在解决该问题之前,必须先了解下JAVA堆内存是怎么工作的. 先看下JAVA堆内存是如何划分的,如图: JVM内存划分为 ...
随机推荐
- XShell转发数据库端口
隧道添加 源主机为本地 目标主机为需要转发的主机
- idea【快捷键】
ctrl+shift+a:全局搜索 IDEA 的操作和设置 shift+f6:重命名 ctrl+alt+m:提取方法 ctrl+alt+l:格式化代码 ctrl+alt+o:优化导入类和包 ctrl+ ...
- jQuery-3.事件篇---自定义事件
jQuery自定义事件之trigger事件 众所周知类似于mousedown.click.keydown等等这类型的事件都是浏览器提供的,通俗叫原生事件,这类型的事件是需要有交互行为才能被触发. 在j ...
- 【EMV L2】终端验证结果(Terminal Verification Results,TVR)
终端验证结果,Terminal Verification Results(TVR),Tag95,5bytes: 记录交易过程中,数据认证.处理限制.持卡人验证.终端风险管理.行为分析以及联机处理的结果 ...
- python excel 读写
python操作Excel读写--使用xlrd xlwt python中使用xlrd.xlwt操作excel表格详解
- Java内省机制
转自: https://blog.csdn.net/hahalzb/article/details/5972421 1.java内省机制其实通俗的理解为,对自身的进行一个扫描,这个扫描的对象就是我们普 ...
- .Net中World转PDF
using System;using System.Collections.Generic;using System.Linq;using System.Web;using Aspose.Words; ...
- asp.net开启多线程异步处理
protected void Page_Load(object sender, EventArgs e) { if (!IsPostBack) { try { Thread categoryThrea ...
- 关于jstl中碰到的Property 'username' not found on type java.lang.String异常
在jstl的forEach循环的时候总是有异常,刚开始以为是把类的属性名打错了,因为显示的是Property not found,但就算从类文件里面复制属性名过来依然显示的是Property not ...
- 踩坑 —— Eclipse MAVEN编译
一.踩坑 1.昨天download了Netty和SOFARPC工程的源码,Eclipse编译的时候报错了,信息如下: Plugin execution not covered by lifecycle ...