使用 docker 进行 ElasticSearch + Kibana 集群搭建

在Docker容器中运行Elasticsearch Kibana和Cerebro

机器信息

10.160.13.139
10.160.9.162
10.160.11.171

1. 安装docker和docker compose

安装docker

# 安装 docker 社区版
yum -y install docker-ce docker-ce-cli containerd.io
# 启动 Docker
systemctl start docker
# 查看 docker 版本
docker version
# 开机启动 Docker
systemctl enable docker

安装docker-compose

sudo curl -L https://github.com/docker/compose/releases/download/1.27.4/docker-compose-`uname -s`-`uname -m` \
> /usr/local/bin/docker-compose
# 添加执行权限
sudo chmod +x /usr/local/bin/docker-compose
# 验证
docker-compose --version
docker-compose version 1.27.4, build 40524192

2. 下载镜像

docker pull elasticsearch:7.10.1
docker pull kibana:7.10.1

3. 将 kibana 上传到私有harbor

[@master software]# docker images |grep kibana
kibana                           7.10.1              3e014820ee3f        3 weeks ago         992MB
[@master software]# docker tag 3e014820ee3f www.test.com/elk/kibana:7.10.1
[@master software]# docker login www.test.com
Username: song
Password:
Login Succeeded
[@master software]# docker push www.test.com/elk/kibana:7.10.1
The push refers to repository [www.test.com/elk/kibana]
00cfe116c0b3: Pushed
20889a7593bc: Pushed
d1409dea493d: Pushed
3448a6e61306: Pushed
f3a4630cb1b0: Pushed
0e3f51adbdfe: Pushed
7bb6eb357f15: Pushed
0263a48f9a3a: Pushed
5acb536ef82f: Pushed
58d51bd5abeb: Pushed
e116279745ca: Pushed
cc33c1bea858: Pushed
5ec5e9c8cf6f: Pushed
291f6e44771a: Pushed
7.10.1: digest: sha256:1731793b7f3e453c65ebaf92ec0b55f4029310ba8abae9e04753a4680dd8210b size: 3247
[@master software]#

4. 将 elasticsearch 上传到私有harbor

[@master software]# docker images |grep elastic
elasticsearch                    7.10.1              558380375f1a        3 weeks ago         774MB
[@master software]# docker tag 558380375f1a www.test.com/elk/elasticsearch:7.10.1
[@master software]# docker push www.test.com/elk/elasticsearch:7.10.1
The push refers to repository [www.test.com/elk/elasticsearch]
d6208dd58075: Pushed
9117213e7470: Pushed
b662ef6f4fc1: Pushed
3f481e912123: Pushed
b456348ecd84: Pushed
e2834bada95f: Pushed
8f84abff69b8: Pushed
911490c8ab00: Pushed
291f6e44771a: Mounted from elk/kibana
7.10.1: digest: sha256:e9a1fe65f68b2d2b9583287d1190f67f23af08582eac4d2a8dc342e4219c7306 size: 2202
[@master software]#

5 .操作系统优化

操作系统以及JVM调优主要是针对安装elasticsearch的机器。对于操作系统，需要调整几个内核参数

vim /etc/sysctl.conf
# fs.file-max主要是配置系统最大打开文件描述符数，建议修改为655360或者更高
fs.file-max=655360
# vm.max_map_count影响Java线程数量，用于限制一个进程可以拥有的VMA(虚拟内存区域)的大小
# 系统默认是65530，建议修改成262144或者更高。
vm.max_map_count = 262144
sysctl -p
sysctl -a

另外，还需要调整进程最大打开文件描述符(nofile)、最大用户进程数(nproc)和最大锁定内存地址空间(memlock)，

vim /etc/security/limits.conf
*        soft    nproc           20480
*        hard    nproc           20480
*        soft    nofile          65536
*        hard    nofile          65536
*        soft    memlock         unlimited
*        hard    memlock         unlimited
# 备注： * 代表Linux所有用户名称（比如es），保存，退出，重新登录才可生效
# 查看
ulimit -a

最后，还需要修改

# /etc/security/limits.d/90-nproc.conf（centos6.x系统）
# /etc/security/limits.d/20-nproc.conf（centos7.x系统）

vim /etc/security/limits.d/20-nproc.conf
*          soft    nproc     4096
# 修改为：
*          soft    nproc     20480

或者直接删除/etc/security/limits.d/20-nproc.conf文件也行。

6. JVM调优

JVM调优主要是针对elasticsearch的JVM内存资源进行优化，elasticsearch的内存资源配置文件为jvm.options，7.1下载的默认设置是1GB

从ES 6开始，只支持64位的JVM（配置config/jvm.options）

配置的建议

• 将内存Xms和Xmx设置成一样，避免heap resize时引发停顿
• Xmx设置不要超过物理内存的50%，单个节点上，最大内存建议不要超过32G内存
• 生产环境，JVM必须使用Server模式
• 关闭JVM Swapping

vim /usr/local/elasticsearch/config/jvm.options
-Xms30g
-Xmx30g

7. 安装 elasticsearch

你可以先新建一个 elasticsearch 文件夹。然后在这个文件夹中，新建一个 docker-compose.yml 文件。

运行docker-compose，本地构建更高效的开发环境，更直观地了解 Elasticsearch 分布式特性

集成Cerebro，方便查看集群状态

mkdir -p /search/odin/elasticsearch/{data01,data02,data03}

mkdir -p /Users/song/study/elasticsearch/plugins/ik

# ES_JAVA_OPTS="-Xms64m -Xmx512m" 内存限制，最小64m，最大512m，可根据需要自行调整，除此之外其他的配置不用做修改

cat /search/odin/elasticsearch/docker-compose.yml

version: '2.2'

services:
  cerebro:
    image: lmenezes/cerebro:0.8.3
    container_name: cerebro
    ports:
      - "9000:9000"
    command:
      - -Dhosts.0.host=http://elasticsearch:9200
    networks:
      - elastic
  kibana:
    image: kibana:7.10.1
    container_name: kibana7
    environment:
      - I18N_LOCALE=zh-CN
      - XPACK_GRAPH_ENABLED=true
      - TIMELION_ENABLED=true
      - XPACK_MONITORING_COLLECTION_ENABLED="true"
    ports:
      - "5601:5601"
    networks:
      - elastic
  elasticsearch:
    image: elasticsearch:7.10.1
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
      - /Users/song/study/elasticsearch/plugins/ik:/usr/share/elasticsearch/plugins/ik
    ports:
      - 9200:9200
      - 9300:9300
    networks:
      - elastic
  elasticsearch2:
    image: elasticsearch:7.10.1
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
      - /Users/song/study/elasticsearch/plugins/ik:/usr/share/elasticsearch/plugins/ik
    ports:
      - 9201:9201
      - 9301:9301
    networks:
      - elastic
  elasticsearch3:
    image: elasticsearch:7.10.1
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
    ulimits:
      nproc: 65535
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
      - /Users/song/study/elasticsearch/plugins/ik:/usr/share/elasticsearch/plugins/ik
    ports:
      - 9202:9202
      - 9302:9302
    networks:
      - elastic
  elastichd:
    image: containerize/elastichd:latest
    container_name: elasticsearch-hd
    networks:
      - elastic
    ports:
      - 9800:9800
    depends_on:
      - "elasticsearch"
    links:
      - "elasticsearch:demo"

volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  elastic:
    driver: bridge

运行docker-compose，安装Elasticsearch

启动

screen
docker-compose up

访问测试

curl -X GET "localhost:9200/_cat/nodes?v&pretty"

停止容器

docker-compose down

停止容器并且移除数据

docker-compose down -v

删除单个容器

$docker rm 容器名/容器ID
-f, –force=false;
-l, –link=false Remove the specified link and not the underlying container;
-v, –volumes=false Remove the volumes associated to the container

删除所有容器

$docker rm `docker ps -aq`

停止、启动、杀死、重启一个容器

$ docker stop 容器名/容器ID
$ docker start 容器名/容器ID
$ docker kill 容器名/容器ID
$ docker restart 容器名/容器ID

显示容器资源使用统计

docker stats 容器ID