docker-compose安装EFK

一、环境

IP	系统	配置	版本
192.168.10.100	Centos7.9	2核4G	Docker Compose version v2.19.1、EFK-7.17.11

EFK版本是试用版本

二、安装docker环境

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
yum makecache fast
yum -y install docker-ce

cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors":["https://pft7f97f.mirror.aliyuncs.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn","https://dockerhub.azk8s.cn","http://hub-mirror.c.163.com"]
}
EOF

systemctl daemon-reload
systemctl start docker

[root@efk efk]# docker compose version
Docker Compose version v2.19.1

三、下载EFK相关镜像

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.17.11
docker pull docker.elastic.co/kibana/kibana:7.17.11
docker pull docker.elastic.co/beats/filebeat:7.17.11

[root@efk efk]# docker images
REPOSITORY                                      TAG       IMAGE ID       CREATED       SIZE
docker.elastic.co/beats/filebeat                7.17.11   b4bef40e4a4a   3 weeks ago   268MB
docker.elastic.co/elasticsearch/elasticsearch   7.17.11   0f404e39b5e6   3 weeks ago   630MB
docker.elastic.co/kibana/kibana                 7.17.11   ff2a71cd3986   3 weeks ago   798MB

四、编辑filebeat.yaml文件

[root@efk efk]# cat filebeat.yaml
filebeat.inputs:
- type: log
  paths:
    - '/usr/share/filebeat/logs/*'

processors:
  - decode_json_fields:
      fields: ["message"]
      target: ""
      overwrite_keys: true

output.elasticsearch:
  hosts: ["http://192.168.10.100:9200"]
  indices:
    - index: "filebeat-%{+yyyy.MM.dd}"

setup.kibana:
  host: "http://192.168.10.100:5601"

logging.json: true
logging.metrics.enabled: false

五、部署EFK系统

5.1 创建数据目录

mkdir /data/efk/es/data/nodes -p
chmod -R 777 /data/efk

5.2 编辑docker-compose.yaml文件

[root@efk efk]# cat docker-compose.yml
version: '3.3'

services:

  elasticsearch:
    image: "docker.elastic.co/elasticsearch/elasticsearch:7.17.11"
    container_name: elasticsearch
    restart: always
    environment:
    - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    - "discovery.type=single-node"
    - "cluster.name=myes"
    - "node.name=jeven"
    # - xpack.security.enabled: "false"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    networks:
      myefk:
        ipv4_address: 172.29.120.10
        aliases:
        - es
        - jeven
    ports:
    - "9200:9200"
    - "9300:9300"
    volumes:
    - /data/efk/es/data/:/usr/share/elasticsearch/data

  kibana:
    image: "docker.elastic.co/kibana/kibana:7.17.11"
    restart: always
    environment:
      # 注意这里的配置，否则会导致kibana页面不能打开
      ELASTICSEARCH.URL: http://192.168.10.100:9200
      ELASTICSEARCH.HOSTS: '["http:/192.168.10.100:9200"]'
      I18N_LOCALE: zh-CN
    networks:
      myefk:
        ipv4_address: 172.29.120.20
        aliases:
          - kibana
          - kib
    ports:
    - "5601:5601"
    links:
    - "elasticsearch"

  filebeat:
    image: "docker.elastic.co/beats/filebeat:7.17.11"
    restart: always
    networks:
      myefk:
        ipv4_address: 172.29.120.30
        aliases:
          - filebeat
          - fb
    user: root
    command: ["--strict.perms=false"]
    volumes:
    - /data/efk/filebeat.yaml:/usr/share/filebeat/filebeat.yml
    - /var/lib/docker:/var/lib/docker:ro
    - /var/run/docker.sock:/var/run/docker.sock
    links:
    - "elasticsearch"
    - "kibana"

networks:
  myefk:
    driver: bridge
    ipam:
      config:
        - subnet: 172.29.120.0/24

5.3 运行EFK

[root@efk efk]# docker compose up -d
[+] Running 4/4
  Network efk_myefk         Created                                                                                                                                                                                                   0.3s
  Container elasticsearch   Started                                                                                                                                                                                                   0.4s
  Container efk-kibana-1    Started                                                                                                                                                                                                   0.8s
  Container efk-filebeat-1  Started

# 关闭命令为 docker compose down

[root@efk efk]# docker compose ps
NAME                IMAGE                                                   COMMAND                  SERVICE             CREATED             STATUS              PORTS
efk-filebeat-1      docker.elastic.co/beats/filebeat:7.17.11                "/usr/bin/tini -- /u…"   filebeat            27 minutes ago      Up 27 minutes
efk-kibana-1        docker.elastic.co/kibana/kibana:7.17.11                 "/bin/tini -- /usr/l…"   kibana              27 minutes ago      Up 27 minutes       0.0.0.0:5601->5601/tcp, :::5601->5601/tcp
elasticsearch       docker.elastic.co/elasticsearch/elasticsearch:7.17.11   "/bin/tini -- /usr/l…"   elasticsearch       27 minutes ago      Up 27 minutes       0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 0.0.0.0:9300->9300/tcp, :::9300->9300/tcp

5.4 查看efk容器日志

[root@efk efk]# docker compose logs |head
elasticsearch  | {"type": "server", "timestamp": "2023-07-19T08:49:09,038Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "myes", "node.name": "jeven", "message": "loaded module [aggs-matrix-stats]" }
elasticsearch  | {"type": "server", "timestamp": "2023-07-19T08:49:09,038Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "myes", "node.name": "jeven", "message": "loaded module [analysis-common]" }
elasticsearch  | {"type": "server", "timestamp": "2023-07-19T08:49:09,038Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "myes", "node.name": "jeven", "message": "loaded module [constant-keyword]" }
elasticsearch  | {"type": "server", "timestamp": "2023-07-19T08:49:09,054Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "myes", "node.name": "jeven", "message": "loaded module [frozen-indices]" }

5.5 测试访问

[root@efk efk]# curl 192.168.10.100:9200
{
  "name" : "jeven",
  "cluster_name" : "myes",
  "cluster_uuid" : "-y4gQ2IvQ_CohEPfppPnSw",
  "version" : {
    "number" : "7.17.11",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "eeedb98c60326ea3d46caef960fb4c77958fb885",
    "build_date" : "2023-06-23T05:33:12.261262042Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

六、访问Kibana服务

6.1 页面访问kibana进入首页

http://192.168.10.100:5601

6.2 查看日志信息

1.进入索引管理界面
选左侧打开目录：Managerment---stack managrment ---数据---索引管理

2.查看filebeta索引信息

3.创建索引

选择：索引模式---创建索引---设置索引名称--索引时间戳字段--创建索引

4.搜索日志信息

在主页，选择discover模块位置，根据字段可搜索日志信息

5.查看日志文件信息

Observability——日志，点击进入

本次搭建引用文章：https://cloud.tencent.com/developer/article/2210662

出现2个问题：

　　1./data/efk/es目录的权限问题，我都改成了777

2.kibana页面不能访问问题，修改了docker-compose.yaml文件中，下面2个字段：

      ELASTICSEARCH.URL: http://192.168.10.100:9200
      ELASTICSEARCH.HOSTS: '["http:/192.168.10.100:9200"]'