ELK套件部署

前言

经过两周的不断碰壁，版本的选择

最终选择ELK的7.6.1套餐

因为我所需要的的警报插件sentinl也才跟新到7.6.1

运行环境：centos7

需要开放的端口：5601，9200，514（udp，tcp）

需要的安装包

https://artifacts.elastic.co/downloads/kibana/kibana-7.6.1-x86_64.rpm

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.1-x86_64.rpm

https://artifacts.elastic.co/downloads/logstash/logstash-7.6.1.rpm

https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.6.1-x86_64.rpm

https://github.com/lmangani/sentinl/releases/download/7.6.1/sentinl-v7.6.1.zip

自己用迅雷下载吧

开始

推荐在根目录下的opt文件夹下创建一个elk目录存放rpm安装包

一、安装Elasticsearch

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
rpm --install elasticsearch-7.6.1-x86_64.rpm

设置开机自起并启动

systemctl daemon-reload
systemctl enable elasticsearch
systemctl start elasticsearch

启动后自行查看状态

修改配置文件并重启

vim /etc/elasticsearch/elasticsearch.yml

#集群名称
cluster.name: test-el
#节点名称
node.name: node-1
#监听IP
network.host: 0.0.0.0
#初始化设置
cluster.initial_master_nodes: ["node-1", "node-2"]

systemctl restart elasticsearch

二、安装kibana

rpm -ivh kibana-7.6.1-x86_64.rpm

设置开机自起并启动

systemctl daemon-reload
systemctl enable kibana
systemctl start kibana

修改配置文件并重启

vim /etc/kibana/kibana.yml

#服务端口
server.port: 5601
#监听地址
server.host: "0.0.0.0"
#elasticsearch 的地址+端口
elasticsearch.hosts: ["http://10.15.5.225:9200"]
#语言
i18n.locale: "zh-CN"

systemctl restart kibana

三、安装logstash

rpm -ivh logstash-logstash-7.6.1.rpm
systemctl daemon-reload
systemctl enable logstash
systemctl start logstash

修改配置文件并重启

vim /etc/logstash/conf.d/symantec.conf

input {
    file {
        path => ["/var/log/symantec/*.log"]
        start_position => "beginning"
    }
 }
output {
        elasticsearch {
                hosts => ["172.26.130.200:9200"]
                index => "symentec_log"
        }
}

systemctl restart logstash

四、配置rsyslog日志服务器

配置文件地址/etc/rsyslog.conf

就以下几个地方需要改的，存放路径按照第三步的logstash的配置文件里面的写法

# Provides UDP syslog reception
$ModLoad imudp  # 引用udp协议的模块
$UDPServerRun 514  # 设置udp协议使用端口

# Provides TCP syslog reception
$ModLoad imtcp  # 引用tcp协议的模块
$InputTCPServerRun 514  # 设置tcp协议使用端口

$template Remote,"/var/log/%$YEAR%-%$MONTH%-%$DAY%/%fromhost-ip%.log" # 设置远程日志存放路径和文件格式
:fromhost-ip, !isequal, "127.0.0.1" ?Remote  # 如果是本机日志则不记录

重启rsyslog

systemctl restart rsyslog

五、使用filebeat接收rsyslog的日志

安装

下载好rpm包后直接安装

rpm -vi filebeat-7.6.1-x86_64.rpm

编辑配置

vim /etc/filebeat/filebeat.yml

output.elasticsearch:
  hosts: ["<es_url>"]
  # username: "elastic"
  # password: "<password>"
setup.kibana:
  host: "<kibana_url>"

启用system模块,并编辑配置

sudo filebeat modules enable system

vim /etc/filebeat/filebeat.yml

filebeat.inputs:

- type: log
  paths:
    - /var/log/symantec/*.log

setup.kibana:
  host: "localhost:5601"

output.elasticsearch:
  hosts: ["localhost:9200"]

启动 Filebeat

filebeat setup
service filebeat start

启动后就可以在页面接收到数据了

六、安装sentinl警告插件

先将zip包放在/opt/elk/下

cd到/usr/share/kibana

注意一点要在这个目录下

执行

[root@node2 kibana]# /usr/share/kibana/bin/kibana-plugin install file:///opt/elk/sentinl-v7.6.1.zip  --allow-root
Found previous install attempt. Deleting...
Attempting to transfer from file:///opt/elk/sentinl-v7.6.1.zip
Transferring 146880856 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Plugin installation complete

[root@node2 kibana]# systemctl restart kibana
[root@node2 kibana]# systemctl status kibana
● kibana.service - Kibana
   Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: disabled)
   Active: active (running) since 一 2020-11-09 15:34:30 CST; 2min 4s ago
 Main PID: 21387 (node)
    Tasks: 22
   CGroup: /system.slice/kibana.service
           ├─21387 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml
           └─21535 /usr/share/kibana/node/bin/node --no-warnings --max-http-header-size=65536 /usr/share/kibana/node_modules/thread-loader/dist/worker.js 20

11月 09 15:34:48 node2 kibana[21387]: {"type":"log","@timestamp":"2020-11-09T07:34:48Z","tags":["info","savedobjects-service"],"pid":21387,"message":"Starting saved objects migrations"}
11月 09 15:34:48 node2 kibana[21387]: {"type":"log","@timestamp":"2020-11-09T07:34:48Z","tags":["info","savedobjects-service"],"pid":21387,"message":"Detected mapping change i...l-watcher\""}
11月 09 15:34:48 node2 kibana[21387]: {"type":"log","@timestamp":"2020-11-09T07:34:48Z","tags":["info","savedobjects-service"],"pid":21387,"message":"Creating index .kibana_2."}
11月 09 15:34:48 node2 kibana[21387]: {"type":"log","@timestamp":"2020-11-09T07:34:48Z","tags":["info","savedobjects-service"],"pid":21387,"message":"Migrating .kibana_1 saved...o .kibana_2"}
11月 09 15:34:48 node2 kibana[21387]: {"type":"log","@timestamp":"2020-11-09T07:34:48Z","tags":["info","savedobjects-service"],"pid":21387,"message":"Pointing alias .kibana to .kibana_2."}
11月 09 15:34:48 node2 kibana[21387]: {"type":"log","@timestamp":"2020-11-09T07:34:48Z","tags":["info","savedobjects-service"],"pid":21387,"message":"Finished in 650ms."}
11月 09 15:34:48 node2 kibana[21387]: {"type":"log","@timestamp":"2020-11-09T07:34:48Z","tags":["info","plugins-system"],"pid":21387,"message":"Starting [22] plugins: [taskManager,siem,inf...
11月 09 15:34:51 node2 kibana[21387]: {"type":"log","@timestamp":"2020-11-09T07:34:51Z","tags":["info","optimize"],"pid":21387,"message":"Optimizing and caching bundles for core, graph, mo...
11月 09 15:34:53 node2 kibana[21387]: Browserslist: caniuse-lite is outdated. Please run next command `npm update`
11月 09 15:34:55 node2 kibana[21387]: Browserslist: caniuse-lite is outdated. Please run next command `npm update`

看到这里爆红不要紧张，等待一会就好了，再次打开kibana页面插件就出来了！