CentOS7下使用ELK搭建日志集中分析平台

应用场景：

ELK实际上是三个工具的集合，ElasticSearch + Logstash + Kibana，这三个工具组合形成了一套实用、易用的监控架构，很多公司利用它来搭建可视化的海量日志分析平台。

官网下载地址：https://www.elastic.co/downloads

Elasticsearch:

　　是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java开发的，并作为Apache许可条款下的开放源码发布，是当前流行的企业级搜索引擎。设计用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。

Logstash:

　　用于管理日志和事件的工具，你可以用它去收集日志、转换日志、解析日志并将他们作为数据提供给其它模块调用，例如搜索、存储等。

Kibana:

　　是一个优秀的前端日志展示框架，它可以非常详细的将日志转化为各种图表，为用户提供强大的数据可视化支持。　　　

Filebeat:

类似于“agent”装在被监控端上（数据源），用来实时收集日志文件数据。

安装环境：

software	ip	system
Filebeat 5.6	192.168.1.101	CentOS 7.4
Filebeat 5.6	192.168.1.102	CentOS 7.4
Filebeat 5.6	192.168.1.103	CentOS 7.4
Logstash 5.6	192.168.1.101	CentOS 7.4
ElasticSearch 5.6	192.168.1.101	CentOS 7.4
ElasticSearch 5.6	192.168.1.102	CentOS 7.4
ElasticSearch 5.6	192.168.1.103	CentOS 7.4
Kibana 5.6	192.168.1.101	CentOS 7.4

软件包：

jdk-8u65-linux-x64.tar.gz

epel-release-7-11.noarch.rpm

elasticsearch-5.6.12.rpm

logstash-5.6.12.rpm

kibana-5.6.12-x86_64.rpm

filebeat-5.6.12-x86_64.rpm

安装步骤：

安装java

[root@mydb1 ~]# mkdir -p /usr/java

[root@mydb1 ~]# cp /soft/jdk-8u65-linux-x64.tar.gz /usr/java

[root@mydb1 ~]# cd /usr/java

[root@mydb1 java]# tar -xvf jdk-8u65-linux-x64.tar.gz

[root@mydb1 java]# rm -rf jdk-8u65-linux-x64.tar.gz

[root@mydb1 ~]# vim .bash_profile

JAVA_HOME=/usr/java/jdk1.8.0_65

CLASSPATH=$JAVA_HOME/lib/

PATH=$JAVA_HOME/bin:$PATH

export PATH JAVA_HOME CLASSPATH

[root@mydb1 ~]# . .bash_profile

[root@mydb1 ~]# java -version

java version "1.8.0_65"

Java(TM) SE Runtime Environment (build 1.8.0_65-b17)

Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)

mydb2,mydb3执行同样的步骤

安装elasticsearch

[root@mydb1 soft]# rpm -ivh elasticsearch-5.6.12.rpm

编辑配置文件

[root@mydb1 ~]# vim /etc/elasticsearch/elasticsearch.yml

启动服务

[root@mydb1 ~]# systemctl daemon-reload

[root@mydb1 ~]# systemctl start elasticsearch

[root@mydb1 ~]# systemctl status elasticsearch

● elasticsearch.service - Elasticsearch

Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)

Active: active (running) since 二 2018-10-30 04:19:05 CST; 8s ago

Docs: http://www.elastic.co

Process: 41715 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)

Main PID: 41717 (java)

CGroup: /system.slice/elasticsearch.service

└─41717 /bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch ...

10月 30 04:19:05 mydb1 systemd[1]: Starting Elasticsearch...

10月 30 04:19:05 mydb1 systemd[1]: Started Elasticsearch.

10月 30 04:19:05 mydb1 elasticsearch[41717]: OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you ...Threads=N

Hint: Some lines were ellipsized, use -l to show in full.

[root@mydb1 ~]# systemctl enable elasticsearch

查看端口状态

配置集群中的另外两个节点es-2和es-3

es集群的相关查询

查询集群的健康状态

ES集群安装结束

安装Kibana

[root@mydb1 soft]# rpm -ivh kibana-5.6.12-x86_64.rpm

编辑配置文件

启动kibana

[root@mydb1 ~]# systemctl start kibana

[root@mydb1 ~]# systemctl status kibana

[root@mydb1 ~]# systemctl enable kibana

添加Nginx反向代理

[root@mydb1 soft]# rpm -ivh epel-release-7-11.noarch.rpm

[root@mydb1 soft]# yum install -y nginx httpd-tools

更改配置文件

删除或注释掉文件/etc/nginx/nginx.conf中的一段server{}

新增kibana.conf文件

[root@mydb1 ~]# cd /etc/nginx/conf.d/

[root@mydb1 conf.d]# cat kibana.conf

server {

listen 80;

server_name kibana;

auth_basic "Restricted Access";

auth_basic_user_file /etc/nginx/kibana-user;

location / {

proxy_pass http://192.168.1.101:5601;

proxy_http_version 1.1;

proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection 'upgrade';

proxy_set_header Host $host;

proxy_cache_bypass $http_upgrade;

}

}

生成页面登陆用户认证文件

[root@mydb1 ~]# htpasswd -cm /etc/nginx/kibana-user adrian

New password:

Re-type new password:

Adding password for user adrian

启动Nginx

[root@mydb1 ~]# nginx -t

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conf test is successful

[root@mydb1 ~]# systemctl start nginx

[root@mydb1 ~]# systemctl status nginx

[root@mydb1 ~]# systemctl enable nginx

浏览器登陆kibana

Kibana安装配置完成

安装Logstash

[root@mydb1 soft]# rpm -ivh logstash-5.6.12.rpm

添加配置文件

[root@mydb1 ~]# vim /etc/logstash/conf.d/logstash.conf

input {

beats {

port => "5044"

}

}

filter {

grok {

# 有ID有use

match => [ "message", "(?m)^# User@Host: %{USER:user}\[[^\]]+\] @ (?:(?<clienthost>\S*) )?\[(?:%{IP:clientip})?\]\s+Id:\s%{NUMBER:id:int}\n# Query_time: %{NUMBER:query_time:float}\s+Lock_time: %{NUMBER:lock_time:float}\s+Rows_sent: %{NUMBER:rows_sent:int}\s+Rows_examined: %{NUMBER:rows_examined:int}\nuse\s(?<dbname>\w+);\nSET\s+timestamp=%{NUMBER:timestamp_mysql:int};\n(?<query>.*)" ]

# 有ID无use

match => [ "message", "(?m)^# User@Host: %{USER:user}\[[^\]]+\] @ (?:(?<clienthost>\S*) )?\[(?:%{IP:clientip})?\]\s+Id:\s%{NUMBER:id:int}\n# Query_time: %{NUMBER:query_time:float}\s+Lock_time: %{NUMBER:lock_time:float}\s+Rows_sent: %{NUMBER:rows_sent:int}\s+Rows_examined: %{NUMBER:rows_examined:int}\nSET\s+timestamp=%{NUMBER:timestamp_mysql:int};\n(?<query>.*)" ]

# 无ID有use

match => [ "message", "(?m)^# User@Host: %{USER:user}\[[^\]]+\] @ (?:(?<clienthost>\S*) )?\[(?:%{IP:clientip})?\]\n# Query_time: %{NUMBER:query_time:float}\s+Lock_time: %{NUMBER:lock_time:float}\s+Rows_sent: %{NUMBER:rows_sent:int}\s+Rows_examined: %{NUMBER:rows_examined:int}\nuse\s(?<dbname>\w+);\nSET\s+timestamp=%{NUMBER:timestamp_mysql:int};\n(?<query>.*)" ]

# 无ID无use

match => [ "message", "(?m)^# User@Host: %{USER:user}\[[^\]]+\] @ (?:(?<clienthost>\S*) )?\[(?:%{IP:clientip})?\]\n# Query_time: %{NUMBER:query_time:float}\s+Lock_time: %{NUMBER:lock_time:float}\s+Rows_sent: %{NUMBER:rows_sent:int}\s+Rows_examined: %{NUMBER:rows_examined:int}\nSET\s+timestamp=%{NUMBER:timestamp_mysql:int};\n(?<query>.*)" ]

}

date {

match => ["timestamp_mysql","UNIX"]

target => "@timestamp"

}

}

output {

elasticsearch {

hosts => ["192.168.1.101:9200"]

index=>"mysql-slow-log-%{+YYYY.MM.dd}"

}

stdout { codec => rubydebug }

}

启动logstash服务

[root@mydb1 ~]# systemctl start logstash

[root@mydb1 ~]# systemctl status logstash

Logstash安装配置完成

安装filebeat

[root@mydb1 soft]# rpm -ivh filebeat-5.6.12-x86_64.rpm

修改配置文件

[root@mydb1 ~]# vim /etc/filebeat/filebeat.yml

filebeat.prospectors:

- input_type: log

paths:

- /app/mysqldata/3306/slow.log

exclude_lines: ['^\# Time']

multiline.pattern: '^\# Time|^\# User'

multiline.negate: true

multiline.match: after

output.logstash:

hosts: ["192.168.1.101:5044"]

[root@mydb1 ~]# systemctl start filebeat

[root@mydb1 ~]# systemctl status filebeat

mydb2,mydb3执行同样的步骤

调整long_query_time值为0，清空/app/mysqldata/3306/slow.log，重启数据库

测试是否正常：

[root@mydb1 ~]# vim /etc/logstash/conf.d/logstash.conf

input {

beats {

port => "5044"

}

}

filter {

}

output {

elasticsearch {

hosts => ["192.168.1.101:9200"]

}

stdout { codec => rubydebug }

}

[root@mydb1 ~]# systemctl start logstash

[root@mydb1 ~]# systemctl status logstash

[root@mydb1 ~]# vim test.sh

while [ 1==1 ];do

sleep 1

send=`date '+%Y-%m-%d %H:%M:%S'`

echo $send ">>hello" >> test.log

done

[root@mydb1 ~]# sh test.sh &

[root@mydb1 ~]# vim /etc/filebeat/filebeat.yml

filebeat.prospectors:

- input_type: log

paths:

- /root/test.log

output.logstash:

hosts: ["l92.168.1.101:5044"]

[root@mydb1 ~]# systemctl start filebeat

[root@mydb1 ~]# systemctl status filebeat