Elasticsearch集群部署和运维命令

Elasticsearch集群部署

下载tar包

在"https://www.elastic.co/cn/downloads/elasticsearch"页面，有 past releases，可以下载以前版本
下载
# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.0-linux-x86_64.tar.gz

解压
# tar -zxvf elasticsearch-7.4.0-linux-x86_64.tar.gz -C /usr/local/

编辑配置系统参数

# vi /etc/sysctl.conf
vm.max_map_count=262144
# /sbin/sysctl -p    --立即生效
# vi /etc/security/limits.conf
* soft nproc 65535
* hard nproc 65535
* soft nofile 65535
* hard nofile 131072

编辑配置文件

# cd elasticsearch-7.4.0/
# vi elasticsearch.yml

 cluster.name: cluster-233
 node.name: node_233_101
 network.host: 10.233.27.103
 network.publish_host: 10.233.27.103
 http.port: 9500
 transport.tcp.port: 9501
 node.master: true
 node.data: true
 path.data: /usr/local/elasticsearch-7.4.0/data
 path.logs: /usr/local/elasticsearch-7.4.0/logs
 path.repo: ["/usr/local/elasticsearch-7.4.0/reposity"]
 # head 插件需要这打开这两个配置
 http.cors.allow-origin: "*"
 http.cors.enabled: true
 http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
 http.cors.allow-headers: "X-Requested-With, Content-Type, Content-Length, X-User"
 http.max_content_length: 200mb
 # 可以选举的主节点
 cluster.initial_master_nodes: ["10.233.27.101","10.233.27.103","10.233.27.104"]
 discovery.seed_hosts: ["10.233.27.101","10.233.27.103","10.233.27.104"]
 gateway.recover_after_nodes: 2
 network.tcp.keep_alive: true
 network.tcp.no_delay: true
 transport.tcp.compress: true
 #集群内同时启动的数据任务个数，默认是2个
 cluster.routing.allocation.cluster_concurrent_rebalance: 16
 #添加或删除节点及负载均衡时并发恢复的线程个数，默认4个
 cluster.routing.allocation.node_concurrent_recoveries: 16
 #初始化数据恢复时，并发恢复线程的个数，默认4个
 cluster.routing.allocation.node_initial_primaries_recoveries: 16
 bootstrap.system_call_filter: false

### 没有节点都配置，node_name、network-host、network.public_host每个节点修改对应的IP

vi jvm.options

-Xms1g   #修改成想要的值
-Xmx1g   #修改成想要的值

建elastic用户

# useradd elastic

### 设置elasticsearch安装目录属主为elastic
# chown -R elastic.elastic elasticsearch-7.4.0

启动elasticsearch

用elastic用户登录
# su - elastic

启动elasticsearch，加-d用守护进程启动
# /usr/local/elasticsearch-7.4.0/bin/elasticsearch -d

查看es状态信息

查看集群健康状态
# curl -XGET "http://10.2.XX.XX:9200/_cluster/health?pretty"
 * "status" : "green"代表健康，"yellow"代表主分片正常副分片不正常，"red"代表主分片不正常

查看集群节点信息
# curl -XGET "http://10.2.2.1:9200/_cat/nodes?v&format=json&pretty"
 * "master" : "*"为主节点，"-"为普通节点

索引信息
# curl -XGET "http://10.2.2.1:9200/_cat/indices?v"
 * health索引健康状态，pri.store.size存储大小

索引信息[:shard]
# curl -XGET "http://10.2.2.1:9200/_cluster/health?pretty&level=indices"
 * "number_of_shards" : 1把索引分为几个分片，"number_of_replicas" : 1每个主分片有几个副本，"unassigned_shards" : 0未分配的shard[磁盘85%或节点宕]

堆Jvm使用率
# curl -XGET "http://10.2.2.1:9200/_cat/nodes?v=true&h=name,node*,heap*"
 * heap.current现使用量 heap.percent百分比 heap.max最大值

[索引-分片]在节点的分配情况
# curl -XGET "http://10.2.2.1:9200/_cat/shards?v=true&s=state"

检查分片分配_cluster/allocation/explain
### 提供检测，给出未分配分片的原因，已分配分片为什么没有rebalance或转移到别的节点的解释
# curl -XGET "http://10.3.4.1:9200/_cluster/allocation/explain?pretty"

对未分配的分片，重试分配
# curl -XPOST "http://10.3.4.1:9200/_cluster/reroute?retry_failed=true"
# curl -XGET "http://10.34.4.153:9200/_cluster/allocation/explain?pretty&filter_path=index,node_allocation_decisions.node_name,node_allocation_decisions.deciders.*"

手动分配分片
 1）第一步，提取node 名称
curl -XGET 'http://10.34.4.153:9200/_nodes/process?pretty=true'
 2）创建脚本，手动分配
#!/bin/bash
NODE="0gniN6q6S4GVuCXtWTRbwQ"
IFS=$'\n'
for line in $(curl -s 'http://10.34.4.153:9200/_cat/shards' | fgrep UNASSIGNED); do
  INDEX=$(echo $line | (awk '{print $1}'))
  SHARD=$(echo $line | (awk '{print $2}'))
  echo $INDEX
  echo  $SHARD
  curl -XPOST 'http://10.34.4.153:9200/_cluster/reroute' -H 'content-Type:application/json' -d '{
     "commands": [
        {
            "allocate": {
                "index": "'$INDEX'",
                "shard": '$SHARD',
                "node": "'$NODE'",
                "allow_primary": true
          }
        }
    ]
  }'
done

安装kibana

kibana作为es的客户端，进行操作；需要安装与es同一版本的包，否则不可用.

下载kibana包
# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.4.0-linux-x86_64.tar.gz

解压
# tar -zxvf kibana-7.4.0-linux-x86_64.tar.gz -C /usr/local

配置文件
# vi /usr/local/kibana-7.4.0-linux-x86_64/config/kibana.yml
server.port: 5601
server.host: "10.23.2.1"
elasticsearch.hosts: ["http://10.23.2.101:9500","http://10.23.2.103:9500","http://10.23.2.104:9500"]

安装logstash

待续...