Kubernetes环境下调整WebLogic JVM基本参数

基于Kubernetes启动WebLogic后，发现JVM的最大heap size一直在700多M左右,通过

kubectl logs 察看pod启动状态,发现日志中并没有-Xms和-Xmx参数.日志部分如下:

[root@k8s-master fluentd-elasticsearch]# kubectl logs helloworld-service-4d72j
.
.
JAVA Memory arguments: -Djava.security.egd=file:/dev/./urandom
.
CLASSPATH=/u01/oracle/wlserver/../oracle_common/modules/javax.persistence_2..jar:/u01/oracle/wlserver/../wlserver/modules/com.oracle.weblogic.jpa21support_1.0.0.0_2-.jar:/usr/java/jdk1..0_101/lib/tools.jar:/u01/oracle/wlserver/server/lib/weblogic_sp.jar:/u01/oracle/wlserver/server/lib/weblogic.jar:/u01/oracle/wlserver/../oracle_common/modules/net.sf.antcontrib_1.1.0.0_1-0b3/lib/ant-contrib.jar:/u01/oracle/wlserver/modules/features/oracle.wls.common.nodemanager_2.0.0..jar:/u01/oracle/wlserver/../oracle_common/modules/com.oracle.cie.config-wls-online_8.1.0..jar:/u01/oracle/wlserver/common/derby/lib/derbyclient.jar:/u01/oracle/wlserver/common/derby/lib/derby.jar:/u01/oracle/wlserver/server/lib/xqrl.jar
.
PATH=/u01/oracle/wlserver/server/bin:/u01/oracle/wlserver/../oracle_common/modules/org.apache.ant_1.9.2/bin:/usr/java/jdk1..0_101/jre/bin:/usr/java/jdk1..0_101/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/java/default/bin:/u01/oracle/oracle_common/common/bin:/u01/oracle/oracle_common/common/bin:/u01/oracle/wlserver/common/bin:/u01/oracle/user_projects/domains/base_domain/bin:/u01/oracle
.
***************************************************
*  To start WebLogic Server, use a username and   *
*  password assigned to an admin-level user.  For *
*  server administration, use the WebLogic Server *
*  console at http://hostname:port/console        *
***************************************************
starting weblogic with Java version:
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) -Bit Server VM (build 25.101-b13, mixed mode)
Starting WLS with line:
/usr/java/jdk1..0_101/bin/java -server   -Djava.security.egd=file:/dev/./urandom -Dweblogic.Name=AdminServer -Djava.security.policy=/u01/oracle/wlserver/server/lib/weblogic.policy  -Dweblogic.ProductionModeEnabled=true   -Djava.endorsed.dirs=/usr/java/jdk1..0_101/jre/lib/endorsed:/u01/oracle/wlserver/../oracle_common/modules/endorsed  -da -Dwls.home=/u01/oracle/wlserver/server -Dweblogic.home=/u01/oracle/wlserver/server     -Dweblogic.utils.cmm.lowertier.ServiceDisabled=true  weblogic.Server

修改yaml文件中的limit参数,发现并不能影响JVM的size，开始怀疑JVM自己根据虚拟机以及启动的container实例的多少自动调整内存，调整replicationcontrol的数目，发现仍然每个进程JVM占用700m.

查阅相关材料，发现Linux Kernel cgroups针对JVM的内存控制机制如下:

• kubernetes switch(-limits)以及docker switch(-m, –memory and –memory-swap)设置都是为了让linux内核决定是否kill掉进程，如果他们消耗比较接近的话,但JVM不会意识到有这个限制。
• JVM的人体工程学机制(JVM ergonomic page)会优化成缺省占用1/4的物理内存，因为JVM并不意识到自己运行在container中，所以一个3G的虚拟机器，JVM最大占用的空间是3*1024/4=768M,正是我们在weblogic管理控制台上看到的jvm最大内存数.
• 如果我们采用limit或者-m 1G的参数，意味着Docker Daemon会限制1G的memory和1G的memory swap,也就是说你进程可以消耗接近2G的内存，不会被kill掉

如果我们想自己控制JVM所占用的heap size,我们可以通过在启动命令行中加入$JAVA_OPTIONS,比如:

$ docker run -d --name mycontainer8g -p 8080:8080 -m 800M -e JAVA_OPTIONS='-Xmx300m' rafabene/java-container:openjdk-env

如果是在kubernetes环境中，可以设置env环境变量把参数传入，因为setDomainEnv中，已经把JAVA_OPTIONS作为启动参数传入命令行。

示例如:

[root@k8s-master ~]# cat rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: helloworld-service
spec:
  replicas:
  template:
    metadata:
      labels:
        weblogic-app: "helloworld"
        version: "0.1"
    spec:
      containers:
      - name: weblogichelloworld
        image: -helloworld:v1
        env:
          - name: "JAVA_OPTIONS"
            value: "-Xms1024m -Xmx1024m"
        ports:
        - containerPort:
---
apiVersion: v1
kind: Service
metadata:
  name: helloworldsvc
  labels:
    weblogic-app: helloworld
spec:
  type: NodePort
  ports:
  - port:
    protocol: TCP
    targetPort:
    name: http
    nodePort:
  selector:
    weblogic-app: helloworld

通过kubectl logs察看pods日志,可以看到-Xms和-Xmx已经被设置，

JAVA Memory arguments: -Djava.security.egd=file:/dev/./urandom
.
CLASSPATH=/u01/oracle/wlserver/../oracle_common/modules/javax.persistence_2..jar:/u01/oracle/wlserver/../wlserver/modules/com.oracle.weblogic.jpa21support_1.0.0.0_2-.jar:/usr/java/jdk1..0_101/lib/tools.jar:/u01/oracle/wlserver/server/lib/weblogic_sp.jar:/u01/oracle/wlserver/server/lib/weblogic.jar:/u01/oracle/wlserver/../oracle_common/modules/net.sf.antcontrib_1.1.0.0_1-0b3/lib/ant-contrib.jar:/u01/oracle/wlserver/modules/features/oracle.wls.common.nodemanager_2.0.0..jar:/u01/oracle/wlserver/../oracle_common/modules/com.oracle.cie.config-wls-online_8.1.0..jar:/u01/oracle/wlserver/common/derby/lib/derbyclient.jar:/u01/oracle/wlserver/common/derby/lib/derby.jar:/u01/oracle/wlserver/server/lib/xqrl.jar
.
PATH=/u01/oracle/wlserver/server/bin:/u01/oracle/wlserver/../oracle_common/modules/org.apache.ant_1.9.2/bin:/usr/java/jdk1..0_101/jre/bin:/usr/java/jdk1..0_101/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/java/default/bin:/u01/oracle/oracle_common/common/bin:/u01/oracle/oracle_common/common/bin:/u01/oracle/wlserver/common/bin:/u01/oracle/user_projects/domains/base_domain/bin:/u01/oracle
.
***************************************************
*  To start WebLogic Server, use a username and   *
*  password assigned to an admin-level user.  For *
*  server administration, use the WebLogic Server *
*  console at http://hostname:port/console        *
***************************************************
starting weblogic with Java version:
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) -Bit Server VM (build 25.101-b13, mixed mode)
Starting WLS with line:
/usr/java/jdk1..0_101/bin/java -server   -Djava.security.egd=file:/dev/./urandom -Dweblogic.Name=AdminServer -Djava.security.policy=/u01/oracle/wlserver/server/lib/weblogic.policy  -Dweblogic.ProductionModeEnabled=true -Xms1024m -Xmx1024m  -Djava.endorsed.dirs=/usr/java/jdk1..0_101/jre/lib/endorsed:/u01/oracle/wlserver/../oracle_common/modules/endorsed  -da -Dwls.home=/u01/oracle/wlserver/server -Dweblogic.home=/u01/oracle/wlserver/server     -Dweblogic.utils.cmm.lowertier.ServiceDisabled=true  weblogic.Server

转到WebLogic管理控制台去查看已经设置成功。

参考文档:

https://developers.redhat.com/blog/2017/03/14/java-inside-docker/

https://fabiokung.com/2014/03/13/memory-inside-linux-containers/