Elasticsearch集群架构的部署和调优（一）

[root@es-node1 ~]# mkdir /usr/java
[root@es-node1 ~]# tar zxvf jdk1.8.0_131.tar.gz -C /usr/java/

[root@es-node1 ~]# cp /etc/profile /etc/profile.back
[root@es-node1 ~]# vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_131/
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOMR/bin
[root@es-node1 ~]# source /etc/profile
[root@es-node1 ~]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

[Elasticsearch]

[root@es-node1 ~]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.2.tar.gz

[root@es-node1 ~]# tar zxvf elasticsearch-6.3.2.tar.gz  -C /usr/src/

[root@es-node1 ~]# mv /usr/src/elasticsearch-6.3.2/ /usr/local/elasticsearch

[root@es-node1 ~]# useradd elasticsearch
[root@es-node1 ~]# chown -R elasticsearch:elasticsearch /usr/local/elasticsearch/

【Elasticsearch优化】

为了使得Elasticsearch获得高效稳定的性能，需要对系统和JVM两个方面进行优化

[root@es-node1 ~]# vim  /etc/sysctl.conf

fs.file-max=655360                 #最大打开文件描述符数，建议修改为655360或者更高
vm.max_map_count=262144      #直接影响java线程数量，用来限制一个进程可以用于VM（虚拟内存区域大小）默认为65530，建议修改为262144或者更高

[root@es-node1 ~]#vim /etc/security/limits.conf

* soft nproc
* hard nproc
* soft nofile
* hard nofile
* soft memlock unlimited
* hard memlock unlimited

[root@es-node1 ~]# sed -i '5s#4096#20480#g' /etc/security/limits.d/20-nproc.conf

#JVM调优

JVM调优主要是针对Elasticsearch的JVM内存资源进行优化，elasticsearch的内存资源配置文件为jvm.options

[root@es-node1 ~]# vim /usr/local/elasticsearch/config/jvm.options   #根据服务器内存大小，进行修改合适的值，建议修改服务器物理内存的一半最佳

-Xms1g
-Xmx1g

[root@es-node1 ~]# egrep -v "#|^$" /usr/local/elasticsearch/config/elasticsearch.yml

cluster.name: es_data #配置集群名称
node.name: es_node1 #节点名称
node.master: true #指定了该节点是否有资格选举master，默认为true
node.data: true #指定该节点是否存储索引数据，默认为true，表示为数据存储
path.data: /usr/local/elasticsearch/data #设置索引数据存储的位置
path.logs: /usr/local/elasticsearch/logs #设置日志文件存储的位置
bootstrap.memory_lock: true #设置true用来锁住物理内存
indices.fielddata.cache.size: 50mb #索引字段缓存大小
network.host: 0.0.0.0 #监听地址
http.port: 9200 #Elasticsearch对外提供的http端口
discovery.zen.ping.unicast.hosts: ["192.168.37.134:9300","192.168.37.135:9300","192.168.37.136:9300"] #设置集群中master节点初始列表，可通过这些
节点自动发现新加入集群的节点，这里的9330端口，即为集群交互通信端口
discovery.zen.minimum_master_nodes: 1 #配置当前集群最少的master节点数，默认为1,也就是说，elasticsearch集群中master节点数不能低于次值>，
http.cors.enabled: true #表示开启跨域访问支持，默认为false
http.cors.allow-origin: "*" #表示跨域访问允许的域名地址，，可支持正则表达式，这里“*”表示允许所有域名访问
http.cors.allow-headers: "X-Requested-With,Content-Type, Content-Length, Authorization" #允许跨域访问头部信息

[root@es-node1 config]# vim /usr/local/elasticsearch/config/jvm.options

-Xms2g
-Xmx2g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+DisableExplicitGC
-XX:+AlwaysPreTouch
-server
-Xss1m
-Djava.awt.headless=true
-Dfile.encoding=UTF-
-Djna.nosys=true
-Djdk.io.permissionsUseCanonicalPath=true
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Dlog4j.skipJansi=true
-XX:+HeapDumpOnOutOfMemoryError

[root@es-node1 ~]#mkdir /usr/local/elasticsearch/data/ -p

[root@es-node1 ~]#chown -R elasticsearch:elasticsearch /usr/local/elasticsearch/

[root@es-node1 ~]# su - elasticsearch
[elasticsearch@es-node1 ~]$ /usr/local/elasticsearch/bin/elasticsearch -d

ps:当你的elasticsearch启动的时候，什么错误都没有报，但是就是端口进程没起来·，此时查看日志信息，输出如下，

这是因为目录权限属主属组是root账号，我们只需chown elasticsearch：elasticsearch -R  /usr/local/elasticsearch/即可

tail /usr/local/elasticsearch/logs/es_cluster.log

    at java.nio.file.Files.createAndCheckIsDirectory(Files.java:) ~[?:1.8.0_131]
    at java.nio.file.Files.createDirectories(Files.java:) ~[?:1.8.0_131]
    at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:) ~[elasticsearch-6.3..jar:6.3.]
    at org.elasticsearch.node.Node.<init>(Node.java:) ~[elasticsearch-6.3..jar:6.3.]
    at org.elasticsearch.node.Node.<init>(Node.java:) ~[elasticsearch-6.3..jar:6.3.]
    at org.elasticsearch.bootstrap.Bootstrap$.<init>(Bootstrap.java:) ~[elasticsearch-6.3..jar:6.3.]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:) ~[elasticsearch-6.3..jar:6.3.]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:) ~[elasticsearch-6.3..jar:6.3.]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:) ~[elasticsearch-6.3..jar:6.3.]
    ...  more

 【Elasticsearch图形化工具】

root@es-node1 ~]# yum install -y nodejs npm git   （ps：如果yum不能正常安装npm和nodejs的话，采用源码吧）

源码安装
wget https://nodejs.org/dist/v9.8.0/node-v9.8.0-linux-x64.tar.xz

xz -d node-v9.8.0-linux-x64.tar.xz
 tar xvf node-v9.8.0-linux-x64.tar

 vim /etc/profile

export NODE_HOME=/root/node-v9.8.0-linux-x64/
export PATH=$PATH:$NODE_HOME/bin

[root@es-node1 ~]# npm -v
5.6.
[root@es-node1 ~]# node -v
v9.8.0

[root@es-node1 ~]# cd /usr/local/
[root@es-node1 local]# git clone git://github.com/mobz/elasticsearch-head.git      #安装head插件

[root@es-node1 local]# npm config set registry http://registry.npm.taobao.rog    #采用淘宝源

[root@es-node1 elasticsearch]# npm install     #安装过程中，有点小意外，纠结了好久才找到办法，确实是一个坑

npm ERR!
npm ERR! If you are behind a proxy, please make sure that the
npm ERR! 'proxy' config is set properly.  See: 'npm help config'
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/--22T06_36_25_382Z-debug.log
或者

npm ERR! code ENOTFOUND
npm ERR! errno ENOTFOUND
npm ERR! network request to http://registry.npm.taobao.rog/phantomjs-prebuilt failed, reason: getaddrinfo ENOTFOUND registry.npm.taobao.rog registry.npm.taobao.rog:80
npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly. See: 'npm help config'

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-25T09_04_38_243Z-debug.log
[root@localhost elasticsearch-head]# npm install
npm ERR! code ENOTFOUND
npm ERR! errno ENOTFOUND
npm ERR! network request to http://registry.npm.taobao.rog/grunt-contrib-jasmine failed, reason: getaddrinfo ENOTFOUND registry.npm.taobao.rog registry.npm.taobao.rog:80
npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly. See: 'npm help config'

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-25T09_04_44_521Z-debug.log

解决方法：

npm config get proxy
npm config get https-proxy

npm config set registry http://registry.cnpmjs.org/

npm ERR! code ELIFECYCLE
npm ERR! errno
npm ERR! phantomjs-prebuilt@2.1. install: `node install.js`
npm ERR! Exit status
npm ERR!
npm ERR! Failed at the phantomjs-prebuilt@2.1. install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/--12T07_42_48_334Z-debug.log

解决办法：

# npm install phantomjs-prebuilt@2.1.14 --ignore-scripts

npm install即可

[root@es-node1 elasticsearch]# vim /usr/local/elasticsearch-head/_site/app.js 

[root@es-node1 elasticsearch-head]# nohup npm run start &     #后台启动head插件服务，默认端口为9100

 web浏览器http://192.168.37.134:9100/验证

【其他节点配置】

【192.168.37.135】es-node2

node2节点值作为数据存储的节点角色，不参与主节点选举

#vim  /usr/local/elasticsearch/config/elasticsearch.yml

cluster.name: es_data
node.name: es.node2
node.master: false
node.data: true
path.data: /usr/local/elasticsearch/data/
path.logs: /usr/local/elasticsearch/logs
bootstrap.memory_lock: true
indices.fielddata.cache.size: 50mb
network.host: 0.0.0.0
http.port:
discovery.zen.ping.unicast.hosts: ["192.168.37.134:9300","192.168.37.135:9300","192.168.37.136:9300"]
discovery.zen.minimum_master_nodes:
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: "X-Requested-With,Content-Type, Content-Length, Authorization"

vim /usr/local/elasticsearch/config/jvm.options

-Xms2g
-Xmx2g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+DisableExplicitGC
-XX:+AlwaysPreTouch
-server
-Xss1m
-Djava.awt.headless=true
-Dfile.encoding=UTF-
-Djna.nosys=true
-Djdk.io.permissionsUseCanonicalPath=true
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Dlog4j.skipJansi=true
-XX:+HeapDumpOnOutOfMemoryError

【192.168.37.136】es-node3节点配置（ps：这里的node3与node1配置都是一样的，都有被选举成为master主节点的可能）

cluster.name: es_data
node.name: es.node3
node.master: true
node.data: true
path.data: /usr/local/elasticsearch/data/
path.logs: /usr/local/elasticsearch/logs
bootstrap.memory_lock: true
indices.fielddata.cache.size: 50mb
network.host: 0.0.0.0
http.port:
discovery.zen.ping.unicast.hosts: ["192.168.37.134:9300","192.168.37.135:9300","192.168.37.136:9300"]
discovery.zen.minimum_master_nodes:
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: "X-Requested-With,Content-Type, Content-Length, Authorization"

vim /usr/local/elasticsearch/config/jvm.options

-Xms2g
-Xmx2g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+DisableExplicitGC
-XX:+AlwaysPreTouch
-server
-Xss1m
-Djava.awt.headless=true
-Dfile.encoding=UTF-
-Djna.nosys=true
-Djdk.io.permissionsUseCanonicalPath=true
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Dlog4j.skipJansi=true
-XX:+HeapDumpOnOutOfMemoryError

【总结】

Elasticsearch节点角色类型

在生产中，高并发的场景容易出现脑裂问题，而在Elasticsearch集群亦是如此，Elasticsearch集群中的每个节点都可以成为主节点，存储数据，提供查询服务

而这些事由两个属性参数控制的也就是node,master和node.data

1:这两种组合表示这个节点有资格成为主节点，又要存储数据，如果该节点真的成为主节点的话，那么自己还要存储数据，压力是相当大的，生产不建议这样配置

node.master: true

node.data:true

2:这种组合是数据存储节点，既不参与选举，也不会成为成为主节点，因为没有成为主节点的资格，只能是数据存储节点

node.master：false

node.data：true

3:拥有参与选举主节点的资格，不会存储节点数据，该节点我们称之为master主节点

node.master：true

node.data: false

4:该节点为客户端节点，数据存储和主节点资格两项都没有，只能处理客户端请求，对海量的请求进行均衡

node.master：false

node.data：false

在一个生产集群中我们可以对这些节点的职责进行划分。
建议集群中设置3台以上的节点作为master节点【node.master: true node.data: false】
这些节点只负责成为主节点，维护整个集群的状态。
再根据数据量设置一批data节点【node.master: false node.data: true】
这些节点只负责存储数据，后期提供建立索引和查询索引的服务，这样的话如果用户请求比较频繁，这些节点的压力也会比较大
所以在集群中建议再设置一批client节点【node.master: false node.data: true】
这些节点只负责处理用户请求，实现请求转发，负载均衡等功能。