Nginx filebeat+logstash+Elasticsearch+kibana实现nginx日志图形化展示

filebeat+logstash+Elasticsearch+kibana实现nginx日志图形化展示

 

by:授客  QQ：1033553122

 

测试环境

Win7 64

CentOS-7-x86_64-DVD-1503-01.iso(kibana安装环境)

CentOS 6.5-x86_64(其它软件安装环境)

nginx-1.10.0

filebeat-5.5.2-linux-x86_64.tar.gz

下载地址：

https://pan.baidu.com/s/1dEBkIuH

https://www.elastic.co/downloads/beats/filebeat#ga-release

https://www.elastic.co/start

kibana-5.5.0-linux-x86_64.tar.gz

下载地址：

https://pan.baidu.com/s/1dEBkIuH

https://www.elastic.co/start

logstash-5.5.2.tar.gz

下载地址：

https://pan.baidu.com/s/1dEBkIuH

https://www.elastic.co/downloads/logstash

elasticsearch-5.5.2

下载地址：

https://pan.baidu.com/s/1dEBkIuH

https://www.elastic.co/downloads/elasticsearch#preview-release

安装Nginx

略

Nginx日志配置

http {

include       mime.types;

default_type  application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" $status $request_time $upstream_response_time $request_length $bytes_sent $body_bytes_sent $gzip_ratio $connection_requests "$http_referer" "$http_user_agent" "$http_x_forwarded_for"';

access_log  logs/access.log  main;

运行nginx

安装java

参考文章：

http://blog.sina.com.cn/s/blog_13cc013b50102w01m.html#_Toc438402186

[root@bogon ~]# java -version

java version "1.8.0_65"

64-Bit Server VM (build 25.65-b01, mixed mode)

注意：logstash要求Java 8，不支持java9

https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

安装logstash

# tar -xzvf logstash-5.5.2.tar.gz

# ls

logstash-5.5.2  logstash-5.5.2.tar.gz

# mkdir -p /usr/local/logstash

# mv logstash-5.5.2 /usr/local/logstash/

 

配置logstash

# vim /usr/local/logstash/logstash-5.5.2/logstash.conf

input { stdin {} }

output {

elasticsearch { hosts => ["192.168.1.101:9200"] }

stdout { codec => rubydebug }

}

说明：

input { stdin {} }  表示从标准输入中接收数据

192.168.1.101:9200  分别代表Elasticsearch搜索访问ip和监听端口

stdout { codec => rubydebug }  表示输出到控制台

参考链接：

https://www.elastic.co/guide/en/logstash/current/config-examples.html

运行logstash

# cd /usr/local/logstash/logstash-5.5.2/

# bin/logstash -f logstash.conf

……(略)

The stdin plugin is now waiting for input:

[2017-07-14T03:40:50,373][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

hello world

{

"@timestamp" => 2017-07-13T19:59:53.848Z,

"@version" => "1",

"host" => "0.0.0.0",

"message" => "hello world"

}

说明：启动后，输入上述带背景色内容 hello world，待控制台输出带黄色背景色内容后，在Elasticsearch中执行搜索，如下

GET /logstash-2017.07.13/_search

如上图，能搜索到输入数据，说明成功了

停止运行logstash

按CTRL + D键

参考链接：

https://www.elastic.co/guide/en/logstash/current/first-event.html

安装Elasticsearch

略

安装kibana

# mkdir -p /usr/local/kibana

# tar -xvzf kibana-5.5.0-linux-x86_64.tar.gz

# mv kibana-5.5.0-linux-x86_64 /usr/local/kibana/

参考链接：

https://www.elastic.co/guide/en/kibana/current/targz.html

配置kibana

# cd /usr/local/kibana/kibana-5.5.0-linux-x86_64/config/

# vim kibana.yml

server.host: "192.168.1.104"

elasticsearch.url: "http://192.168.1.101:9200"

参考链接：

https://www.elastic.co/guide/en/kibana/current/settings.html

运行kibana

# cd /usr/local/kibana/kibana-5.5.0-linux-x86_64/

# ./bin/kibana

log   [23:51:04.051] [info][status][plugin:kibana@5.5.0] Status changed from uninitialized to green - Ready

log   [23:51:04.510] [info][status][plugin:elasticsearch@5.5.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch

log   [23:51:04.594] [info][status][plugin:console@5.5.0] Status changed from uninitialized to green - Ready

log   [23:51:04.617] [warning] You're running Kibana 5.5.0 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v5.5.2 @ 192.168.1.101:9200 (192.168.1.101)

log   [23:51:04.674] [info][status][plugin:metrics@5.5.0] Status changed from uninitialized to green - Ready

log   [23:51:04.706] [info][status][plugin:elasticsearch@5.5.0] Status changed from yellow to green - Kibana index ready

log   [23:51:06.992] [info][status][plugin:timelion@5.5.0] Status changed from uninitialized to green - Ready

log   [23:51:07.032] [info][listening] Server running at http://192.168.1.104:5601

log   [23:51:07.037] [info][status][ui settings] Status changed from uninitialized to green - Ready

验证

浏览器中访问：http://192.168.1.104:5601/status

结果发现打不开

解决方法：停止防火墙

# systemctl stop firewalld.service

再次访问

参考链接：

https://www.elastic.co/guide/en/kibana/current/access.html

配置索引模式（index pattern）

要使用Kibana至少需要配置一个索引模式(index pattern)。索引模式用于确认执行搜索和分析的Elasticsearch索引。

窗体顶端

Index name or pattern

配置索引名称，或者索引模式。索引模式允许使用通配符 * 。比如 logstash-*

Time Filter field name

设置时间过滤器，方便在Discover页面中按时间筛选数据

窗体底端

Management -> Index Patterns -> Create Index Pattern，重新设置

参考链接：

https://www.elastic.co/guide/en/kibana/current/connect-to-elasticsearch.html

安装filebeat

# tar -xvzf filebeat-5.5.2-linux-x86_64.tar.gz

# mkdir -p /usr/local/filebeat

# mv filebeat-5.5.2-linux-x86_64 /usr/local/filebeat/

配置

# vim /usr/local/filebeat/filebeat-5.5.2-linux-x86_64/filebeat.yml

配置日志文件路径

如上，可以指定具体的文件名，

- /usr/local/ngnix/logs/access.log

- /usr/local/ngnix/logs/error.log

也可以使用通配符，表示/usr/local/ngnix/logs/目录下，所有.log文件

- /usr/local/ngnix/logs/*.log

配置logstash输出

 

注意：hosts: 后面必须接一个空格，否则会报错

测试配置是否正确

# cd /usr/local/filebeat/filebeat-5.5.2-linux-x86_64/

# ./filebeat -configtest -e

2017/08/17 23:55:32.651228 beat.go:285: INFO Home path: [/usr/local/filebeat/filebeat-5.5.2-linux-x86_64] Config path: [/usr/local/filebeat/filebeat-5.5.2-linux-x86_64] Data path: [/usr/local/filebeat/filebeat-5.5.2-linux-x86_64/data] Logs path: [/usr/local/filebeat/filebeat-5.5.2-linux-x86_64/logs]

2017/08/17 23:55:32.651335 beat.go:186: INFO Setup Beat: filebeat; Version: 5.5.2

2017/08/17 23:55:32.651564 logstash.go:90: INFO Max Retries set to: 3

2017/08/17 23:55:32.652006 outputs.go:108: INFO Activated logstash as output plugin.

2017/08/17 23:55:32.652250 metrics.go:23: INFO Metrics logging every 30s

2017/08/17 23:55:32.662026 publish.go:295: INFO Publisher name: bogon

2017/08/17 23:55:32.698907 async.go:63: INFO Flush Interval set to: 1s

2017/08/17 23:55:32.699214 async.go:64: INFO Max Bulk Size set to: 2048

Config OK

运行filebeat

# ./filebeat -e -c filebeat.yml -d "publish"

参考链接：

https://www.elastic.co/guide/en/beats/filebeat/5.5/filebeat-starting.html

https://www.elastic.co/guide/en/beats/filebeat/5.5/config-filebeat-logstash.html

修改logstash配置

[root@bogon logstash-5.5.2]# vim logstash.conf

input {

beats {

port => "9400"

}

}

filter{

grok {

match => {"message" => "%{IP:remote_addr} - %{USER:remote_user} \[%{HTTPDATE:time_local}\] "%{WORD:method} %{URIPATHPARAM:request} %{DATA:http_version}" %{NUMBER:status:int} %{NUMBER:request_time:float} %{NUMBER:upstream_response_time:float} %{NUMBER:request_length:int} %{NUMBER:bytes_sent:int} %{NUMBER:body_bytes_sent:int} %{DATA:gzip_ratio:float} %{NUMBER:connection_requests:int} "%{DATA:http_referer}" %{QUOTEDSTRING:http_user_agent} %{DATA:http_x_forwarded_for}"}

}

}

output {

elasticsearch { hosts => ["192.168.1.101:9200"] }

stdout { codec => rubydebug }

}

~

~

message对应的日志样例如下：

"192.168.1.101 - - [15/Sep/2017:01:04:51 +0800] "GET /zentaopms/www/theme/default/zh-cn.default.css?v=8.0 HTTP/1.1" 304 0.006 0.006 652 141 0 - 1 "http://192.168.1.102:8080/zentaopms/www/index.php?m=user&f=login&referer=L3plbnRhb3Btcy93d3cvaW5kZXgucGhw" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:55.0) Gecko/20100101 Firefox/55.0" "-"",

测试配置是否正确

# cd /usr/local/logstash/logstash-5.5.2/

# bin/logstash -f logstash.conf --config.test_and_exit

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

Sending Logstash's logs to /usr/local/logstash/logstash-5.5.2/logs which is now configured via log4j2.properties

Configuration OK

[2017-08-31T00:14:15,049][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

运行logstash

说明：如果以--config.reload.automatic方式运行，已经在运行了，修改配置后，会自动重新加载配置，不需要重新运行logstash

# bin/logstash -f logstash.conf --config.reload.automatic

17/08/18 00:53:20.024649 output.go:109: DBG  output worker: publish 323 events

2017/08/18 00:53:20.075676 single.go:140: ERR Connecting error publishing events (retrying): dial tcp 192.168.1.103:9400: getsockopt: no route to host

2017/08/18 00:53:21.109983 single.go:140: ERR Connecting error publishing events (retrying): dial tcp 192.168.1.103:9400: getsockopt: no route to host

2017/08/18 00:53:23.270575 single.go:140: ERR Connecting error publishing events (retrying): dial tcp 192.168.1.103:9400: getsockopt: no route to host

2017/08/18 00:53:27.467576 single.go:140: ERR Connecting error publishing

……

解决方法：防火墙开放端口

#  firewall-cmd --permanent --zone=public --add-port=9400/tcp

success

# firewall-cmd --reload

success

[root@bogon logstash-5.5.2]# bin/logstash -f logstash.conf --config.test_and_exit

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

Sending Logstash's logs to /usr/local/logstash/logstash-5.5.2/logs which is now configured via log4j2.properties

Configuration OK

[2017-09-03T17:56:46,275][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

# bin/logstash -f logstash.conf --config.reload.automatic

 

字段：

参考链接：

https://github.com/elastic/logstash/blob/v1.1.9/patterns/grok-patterns

参考链接：

https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html