ELK收集openstack日志

1、安装jdk

每个openstack服务器需要安装jdk，我安装的版本jdk-7u71-linux-x64.rpm

2、安装、配置Elastic Search

install https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/rpm/elasticsearch/2.3.3/elasticsearch-2.3.3.rpm

让elastic search监听所有IP, 保证其他主机能够访问, 需要手动配置network.host
# echo 'network.host: 0.0.0.0' >> /etc/elasticsearch/elasticsearch.yml

systemctl enable elasticsearch.service
systemctl start elasticsearch.service

安装、配置Log Stash需要收集日志的服务器都安装

添加logstash的yum源:
# rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
# vi /etc/yum.repos.d/logstash.repo
[logstash-2.3]
name=Logstash repository for 2.3.x packages
baseurl=https://packages.elastic.co/logstash/2.3/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

# 通过yum安装logstash
# sudo yum install logstash -y

配置文件:

[root@controller02 conf.d]# pwd
/etc/logstash/conf.d

我的目录分配

[root@controller02 conf.d]# ls
cinder.conf    neutron.conf  nova.conf output.conf

[root@controller02 conf.d]# cat cinder.conf
input {
file{
type => "cinder"
path => ["/var/log/cinder/api.log",
"/var/log/cinder/scheduler.log",
"/var/log/cinder/volume.log"]
}
}
[root@controller02 conf.d]# cat nova.conf
input {
file{
type => "nova"
path => ["/var/log/nova/nova-scheduler.log",
"/var/log/nova/nova-consoleauth.log",
"/var/log/nova/nova-conductor.log",
"/var/log/nova/nova-conductor.log",
"/var/log/nova/nova-api.log"]
}
}
[root@controller02 conf.d]# cat neutron.conf
input {
file{
type => "neutron"
path => ["/var/log/neutron/server.log",
"/var/log/neutron/linuxbridge-agent.log",
"/var/log/neutron/dhcp-agent.log",
"/var/log/neutron/metadata-agent.log"]
}
}
[root@controller02 conf.d]# cat output.conf
output {
elasticsearch{
hosts=>"172.16.21.5:9200"   这个IP是Elastic Search服务器的ip
max_retries=>-1
index => "%{type}-index"
}
}

systemctl enable logstash.service

systemctl start logstash.service

注意: 需要查看一下logstash用户是否有权限访问指定的log文件

usermod -a -G neutron logstash
usermod -a -G nova logstash
usermod -a -G cinder logstash

安装、配置Kibana（在Elastic Search的机器上安装即可）

添加kibana的yum源:
rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
/etc/yum.repos.d/kibana.repo
[kibana-4.5]
name=Kibana repository for 4.5.x packages
baseurl=http://packages.elastic.co/kibana/4.5/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

# 通过yum安装kibana
# yum install kibana -y

systemctl enable kibana.service
systemctl start kibana.service

先可以通过以下命令确认elastic search是否已经获取到logstash上的index数据信息。

WEB页面访问: http://KIBANA_IP:5601

刚进入页面时, 会要求你输入index名字, 可以填入neutron-index, 以及nova-index, 并按CREATE。

测试：