ElasticSearch之系统关键配置

ElasticSearch之系统关键配置

集群名称

在配置文件$ES_HOME/config/elasticsearch.yml中指定，样例如下：

cluster:
    name: logging-prod

或者

cluster.name: logging-prod

节点的名称

在配置文件$ES_HOME/config/elasticsearch.yml中指定，样例如下：

node:
    name: prod-data-2

或者

node.name: prod-data-2

监听IP地址

在配置文件$ES_HOME/config/elasticsearch.yml中指定，样例如下：

network:
    host: 192.168.1.10

或者

network.host: 192.168.1.10

集群发现的配置

在配置文件$ES_HOME/config/elasticsearch.yml中指定，样例如下：

discovery.seed_hosts:
   - 192.168.1.10:9300
   - 192.168.1.11
   - seeds.mydomain.com
   - [0:0:0:0:0:ffff:c0a8:10c]:9301
cluster.initial_master_nodes:
   - master-node-a
   - master-node-b
   - master-node-c

支持IPv4、IPv6、域名、主机名，允许同时指定端口。

路径

路径相关的关键配置，包括：

• 数据的保存路径
• 日志文件的保存路径

数据的保存路径，在配置文件$ES_HOME/config/elasticsearch.yml中使用配置项path.data指定。

日志文件的保存路径，在配置文件$ES_HOME/config/elasticsearch.yml中使用配置项path.logs指定。

配置样例如下：

path:
  data: /var/data/elasticsearch
  logs: /var/log/elasticsearch

或者

path.data: /var/data/elasticsearch
path.logs: /var/log/elasticsearch

临时目录

默认情况下，ElasticSearch使用/tmp作为临时目录。

可以使用环境变量ES_TMPDIR来指定其它路径，命令样例如下：

export ES_TMPDIR=/path/to/tmp
./bin/elasticsearch

启动elasticsearch时假如遇到了如下报错，需要确认临时目录在挂载时是否指定了noexec选项。

• java.lang.UnsatisfiedLinkerError
• failed to map segment from shared object
• failed to allocate closure

上述报错和JNA和libffi相关，ElasticSearch使用这两个完成一些平台相关的业务逻辑。

假如临时目录所在的分区在挂载时指定了noexec选项，并且出于一些原因，不方便切换新的临时目录。

为使JNA和libffi正常工作，可以指定各自的临时目录，操作命令样例如下：

export LIBFFI_TMPDIR=/path/to/tmp
export ES_JAVA_OPTS="-Djava.io.tmpdir=/path/to/temp/dir"
./bin/elasticsearch

JVM的配置

通过环境变量ES_JAVA_OPTS，可以指定JVM参数，如下是命令样例：

ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch

另外一个例子，如下：

ES_JAVA_OPTS="-Xms2g -Xmx2g -Djava.io.tmpdir=/path/to/temp/dir" ./bin/elasticsearch

另外可以通过修改配置文件$ES_HOME/config/jvm.options，增加JVM相关的参数。

当前$ES_HOME/config/jvm.options中默认提供了如下参数。

-XX:+UseG1GC

## JVM temporary directory
-Djava.io.tmpdir=${ES_TMPDIR}

# Leverages accelerated vector hardware instructions; removing this may
# result in less optimal vector performance
20-:--add-modules=jdk.incubator.vector

## heap dumps

# generate a heap dump when an allocation from the Java heap fails; heap dumps
# are created in the working directory of the JVM unless an alternative path is
# specified
-XX:+HeapDumpOnOutOfMemoryError

# exit right after heap dump on out of memory error
-XX:+ExitOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=data

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=logs/hs_err_pid%p.log

## GC logging
-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m

增加堆的参数，样例如下：

-Xms4g
-Xmx4g
-XX:MaxDirectMemorySize=2g

修改参数之后，可以通过日志，观察JVM的参数。

[2023-11-22T00:21:47,743][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [jackie-ubuntu] Java vector incubator API enabled; uses preferredBitSize=256
[2023-11-22T00:21:48,670][INFO ][o.e.n.Node               ] [jackie-ubuntu] version[8.11.1], pid[19483], build[tar/6f9ff581fbcde658e6f69d6ce03050f060d1fd0c/2023-11-11T10:05:59.421038163Z], OS[Linux/5.4.0-166-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21.0.1/21.0.1+12-29]
[2023-11-22T00:21:48,672][INFO ][o.e.n.Node               ] [jackie-ubuntu] JVM home [/home/jackie/software/elasticsearch-8.11.1/jdk], using bundled JDK [true]
[2023-11-22T00:21:48,672][INFO ][o.e.n.Node               ] [jackie-ubuntu] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-18105487545703078353, --add-modules=jdk.incubator.vector, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms1926m, -Xmx1926m, -XX:MaxDirectMemorySize=1009778688, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=tar, --module-path=/home/jackie/software/elasticsearch-8.11.1/lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]

[2023-11-22T00:22:09,195][INFO ][o.e.e.NodeEnvironment    ] [jackie-ubuntu] heap size [1.8gb], compressed ordinary object pointers [true]

也可以通过API，查看JVM的配置。

curl -X GET "https://localhost:9200/_nodes/jvm?pretty" --cacert $ES_HOME/config/certs/http_ca.crt -u "elastic:ohCxPH=QBE+s5=*lo7F9"

执行结果如下：

{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "aKgBu7LgS9a6iPYH8n2JPw" : {
      "name" : "jackie-ubuntu",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1",
      "version" : "8.11.1",
      "transport_version" : 8512001,
      "index_version" : 8500003,
      "component_versions" : {
        "transform_config_version" : 10000099,
        "ml_config_version" : 11000099
      },
      "build_flavor" : "default",
      "build_type" : "tar",
      "build_hash" : "6f9ff581fbcde658e6f69d6ce03050f060d1fd0c",
      "roles" : [
        "data",
        "data_cold",
        "data_content",
        "data_frozen",
        "data_hot",
        "data_warm",
        "ingest",
        "master",
        "ml",
        "remote_cluster_client",
        "transform"
      ],
      "attributes" : {
        "ml.machine_memory" : "4040327168",
        "ml.allocated_processors" : "4",
        "ml.allocated_processors_double" : "4.0",
        "ml.max_jvm_size" : "2021654528",
        "ml.config_version" : "11.0.0",
        "xpack.installed" : "true",
        "transform.config_version" : "10.0.0"
      },
      "jvm" : {
        "pid" : 19483,
        "version" : "21.0.1",
        "vm_name" : "OpenJDK 64-Bit Server VM",
        "vm_version" : "21.0.1+12-29",
        "vm_vendor" : "Oracle Corporation",
        "using_bundled_jdk" : true,
        "start_time_in_millis" : 1700583699298,
        "mem" : {
          "heap_init_in_bytes" : 2021654528,
          "heap_max_in_bytes" : 2021654528,
          "non_heap_init_in_bytes" : 7667712,
          "non_heap_max_in_bytes" : 0,
          "direct_max_in_bytes" : 0
        },
        "gc_collectors" : [
          "G1 Young Generation",
          "G1 Concurrent GC",
          "G1 Old Generation"
        ],
        "memory_pools" : [
          "CodeHeap 'non-nmethods'",
          "Metaspace",
          "CodeHeap 'profiled nmethods'",
          "Compressed Class Space",
          "G1 Eden Space",
          "G1 Old Gen",
          "G1 Survivor Space",
          "CodeHeap 'non-profiled nmethods'"
        ],
        "using_compressed_ordinary_object_pointers" : "true",
        "input_arguments" : [
          "-Des.networkaddress.cache.ttl=60",
          "-Des.networkaddress.cache.negative.ttl=10",
          "-Djava.security.manager=allow",
          "-XX:+AlwaysPreTouch",
          "-Xss1m",
          "-Djava.awt.headless=true",
          "-Dfile.encoding=UTF-8",
          "-Djna.nosys=true",
          "-XX:-OmitStackTraceInFastThrow",
          "-Dio.netty.noUnsafe=true",
          "-Dio.netty.noKeySetOptimization=true",
          "-Dio.netty.recycler.maxCapacityPerThread=0",
          "-Dlog4j.shutdownHookEnabled=false",
          "-Dlog4j2.disable.jmx=true",
          "-Dlog4j2.formatMsgNoLookups=true",
          "-Djava.locale.providers=SPI,COMPAT",
          "--add-opens=java.base/java.io=org.elasticsearch.preallocate",
          "-XX:+UseG1GC",
          "-Djava.io.tmpdir=/tmp/elasticsearch-18105487545703078353",
          "--add-modules=jdk.incubator.vector",
          "-XX:+HeapDumpOnOutOfMemoryError",
          "-XX:+ExitOnOutOfMemoryError",
          "-XX:HeapDumpPath=data",
          "-XX:ErrorFile=logs/hs_err_pid%p.log",
          "-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m",
          "-Xms1926m",
          "-Xmx1926m",
          "-XX:MaxDirectMemorySize=1009778688",
          "-XX:G1HeapRegionSize=4m",
          "-XX:InitiatingHeapOccupancyPercent=30",
          "-XX:G1ReservePercent=15",
          "-Des.distribution.type=tar",
          "--module-path=/home/jackie/software/elasticsearch-8.11.1/lib",
          "--add-modules=jdk.net",
          "--add-modules=ALL-MODULE-PATH",
          "-Djdk.module.main=org.elasticsearch.server"
        ]
      }
    }
  }
}

相关资料

• Important Elasticsearch configuration
• Advanced configuration
• Important system configuration
• Nodes info API
• Ensure JNA temporary directory permits executables
• Discovery
• Bootstrapping a cluster
• Quorum-based decision making