1.ElasticSearch系列之集群部署

第一步：安装JDK

JDK要求jdk1.8+，不安装也可以，ES自带JDK

第二步：系统配置

2.1 禁用交换区

    sudo swapoff -a

2.2 开最大文件数的限制

    编辑文件 /etc/security/limits.conf把nofile设置为65536
    或者执行
    echo  "* soft nofile 65536" >> /etc/security/limits.conf
    echo  "* hard nofile 65536" >> /etc/security/limits.conf
    echo  "elasticsearch soft memlock unlimited" >> /etc/security/limits.conf
    echo  "elasticsearch hard memlock unlimited" >> /etc/security/limits.conf

2.3 设置虚拟内存

    临时生效 sysctl -w vm.max_map_count=262144
    永久生效 编辑/etc/sysctl.conf vm.max_map_count
    验证命令 sysctl vm.max_map_count

2.4 线程数量

    root用户执行 ulimit -u 4096
    或者
    编辑/etc/security/limits.conf把nproc设置为4096
    echo  "* soft nproc 4096" >> /etc/security/limits.conf
    echo  "* hard nproc 4096" >> /etc/security/limits.conf

第三步：创建用户并设置密码

    useradd elasticsearch
    passwd elasticsearch

第四步：安装目录介绍

4.1 创建安装包目录

mkdir -p home/software

4.2 上传压缩包并解压

tar zxvf ES安装包.tar.gz
tar xf elasticsearch-7.6.2-linux-x86_64.tar.gz -C /usr/local
tar xf kibana-7.6.2-linux-x86_64.tar.gz -C /usr/local

4.3 数据日志位置介绍及创建

• ES安装目录 /usr/local
• ES_HOME /usr/local/elasticsearch-7.6.2
• 数据目录 /home/elasticsearch/data
• 日志目录 /home/elasticsearch/logs
• 配置目录 /home/elasticsearch/config

mkdir /home/elasticsearch/data /home/elasticsearch/logs /home/elasticsearch/config

4.4 拷贝config目录

cp /usr/local/elasticsearch-7.6.2/config/* /home/elasticsearch/config

第五步：修改ES基本配置

5.1 进入elasticsearch.yml

vi /usr/local/elasticsearch-7.6.2/config/elasticsearch.yml

5.1 修改elasticsearch.yml配置

# 集群名称
cluster.name: shenjian
# 配置数据目录
path.data: /home/elasticsearch/data
# 配置日志目录
path.logs: /home/elasticsearch/logs
# 启动时是否锁定内存
bootstrap.memory_lock: true
# 节点启动时要加入的集群机器列表，可配置所有节点
discovery.seed_hosts: ["192.168.0.6","192.168.0.7", "192.168.0.8"]
# 初始化可成为主节点的列表
cluster.initial_master_nodes: ["192.168.0.6","192.168.0.7", "192.168.0.8"]

5.1 修改jvm.options配置

-Xms20g
-Xmx20g

# JAVA8 9 GC LOG路径
8:-Xloggc:/home/elasticsearch/logs/gc.log
file=/home/elasticsearch/logs/gc.log

第六步：创建启动脚本

start_elasticsearch.sh

###指定ES的安装目录
ES_HOME="/usr/local/elasticsearch-7.6.2"
###指定ES配置文件的目录
export ES_PATH_CONF="/home/elasticsearch/config"
pid=`ps ax | grep elasticsearch | grep java | head -1 | awk '{print $1}'`
kill -9 $pid
echo "kill -9 $pid success"
ulimit -u 4096
#IP=`ifconfig | grep "inet addr:" |grep 192.168.0| sed '/127/d' | awk '{print $2}' | awk -F : '{print $2}'`
#echo $IP
IP=192.168.0.6
$ES_HOME/bin/elasticsearch -d -Enode.name=HOST${IP} -Enetwork.host=${IP}

第七步：授权并启动

chown -R elasticsearch:ealsticsearch /home/elasticsearch
chmod -x /home/elasticsearch/start_elasticsearch.sh
./start_elasticsearch.sh

第八步：验证集群启动情况

curl http://192.168.0.8:9200/_cat/nodes?pretty

可以看到打印信息
192.168.0.7 10 78  0 0.11 0.05 0.06 dilm * HOST192.168.0.7
192.168.0.8 17 79  1 0.02 0.02 0.05 dilm - HOST192.168.0.8
192.168.0.6 16 87 29 0.77 0.38 0.18 dilm - HOST192.168.0.6

遇到的问题

低版本的Centos抛出的警告

java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed
	at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:341) ~[elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:616) ~[elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:258) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:110) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.2.4.jar:6.2.4]
	at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.2.4.jar:6.2.4]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.2.4.jar:6.2.4]

	解决方案：

	修改elasticsearch.yml 添加一下内容
    bootstrap.memory_lock: false
    bootstrap.system_call_filter: false

java.io.IOException: No route to host

Caused by: java.io.IOException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:715) ~[?:?]
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) ~[netty-transport-4.1.43.Final.jar:4.1.43.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:688) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:600) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:554) ~[?:?]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514) ~[?:?]
	at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050) ~[?:?]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_281]

	192.168.0.6机器出现该问题，查看防火墙，发现与其他两台不一致
	firewall-cmd --state # 查看防火墙状态
    systemctl stop firewalld.service # 关闭防火墙
    systemctl disable firewalld.service # 禁止开机自启动
    ok,解决

欢迎关注公众号算法小生或沈健的技术博客