ELK集群之metricbeat（9）

Metricbeat包的安装及简单使用

        Metricbeat包的安装及简单使用
系统数据采集
  Python -> ES -> Grafana

metricbeat的安装
  metricbeat -> logstash -> ES -> Grafana系统监控
  Metricbeat解决的问题：手动写Python比较麻烦
  yum localinstall metricbeat-7.6.2-x86_64.rpm -y

metric默认配置说明
  /etc/metricbeat/metricbeat.yml
  /etc/metricbeat/modules.d/*.yml

metric配置/etc/metricbeat/metricbeat.yml
metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression
setup.kibana:
output.logstash:
  hosts: ["192.168.238.90:5044"]

metricbeat输出到logstash

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression
setup.kibana:
output.kafka:
  hosts: ["172.17.166.217:9092", "172.17.166.218:9092", "172.17.166.219:9092"]

  topic: 'test2'
  partition.round_robin:
    reachable_only: false

  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000

metricbeat输出到kafka

metricbeat采集系统配置

# Module: system
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.6/metricbeat-module-system.html
- module: system
  period: 60s
  metricsets:
    - network
  interfaces:
    - eth0

- module: system
  period: 10s
  metricsets:
    - cpu
    - load
    - memory
    #- network
    #- process
    #- process_summary
    #- socket_summary
    #- entropy
    #- core
    #- diskio
    #- socket
    #- service
  #process.include_top_n:
  #  by_cpu: 5      # include top 5 processes by CPU
  #  by_memory: 5   # include top 5 processes by memory

- module: system
  period: 1m
  metricsets:
    - filesystem
#    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'

#q
#- module: system
#  period: 15m
#  metricsets:
#    - uptime

#- module: system
#  period: 5m
#  metricsets:
#    - raid
#  raid.mount_point: '/'   

system.yml

        Metricbeat收集cpu内存等信息Grafana展示
Kibana上先清空之前的索引信息

load展示
  metric: avg system.load.1
  metric: avg system.load.5
  metric: avg system.load.15
  groupby: host.name.keyword

  alias: {{host.name.keyword}} {{metric}} {{field}}
  interval: 1m

cpu展示，如果显示异常，清空索引后再观察下
  cpu.idle.pct
  cpu.system.pct
  cpu.user.pct
  cpu.iowait.pct

磁盘使用率
system.filesystem.used.pct

内存使用率的展现
system.memory.actual.used.pct

        Metricbeat收集流量信息Grafana展示
需要配置单独收集某个网卡的流量/etc/metricbeat/modules.d/system.yml
- module: system
  period: 60s
  metricsets:
    - network
  interfaces:
    - eth0

Grafana配置流量信息1m=60s 1Byte=8bit
  avg(system.network.in.bytes)  _value/60*8
  Derivative(avg(system.network.in.bytes))

  alias {{host.name.keyword}} in traffic
  alias {{host.name.keyword}} out traffic

metricbeat modules enable nginx #开启模块

多台metricbeat安装-多台服务器采集
metricbeat安装
  yum localinstall metricbeat-7.6.2-x86_64.rpm -y

metric配置/etc/metricbeat/metricbeat.yml
  拷贝之前一台服务器的

metricbeat配置/etc/metricbeat/modules.d/system.yml
  拷贝之前一台服务器的

启动查看grafana
  systemctl restart metricbeat

        Metricbeat采集Nginx信息Grafana展示
Nginx编译需要加以下选项
  --with-http_stub_status_module

Nginx配置新增
        location /sjgstatus {
            stub_status on;
            access_log off;
        allow 192.168.0.0/16;
        deny all;
        }

Metricbeat支持nginx
  metricbeat modules enable nginx

测试能否获取数据
  curl xxx/sjgstatus

Grafana配置nginx请求情况
  avg(requests) _value / 60
  Derivative
  term by : service.address.keyword
  alias: {{service.address.keyword}} accept persecond

模拟数据
  while true; do  curl 192.168.238.90; curl 192.168.238.90/test; sleep 1; done

        Metricbeat采集多台Nginx数据
安装Nginx，跟之前一样的配置

Metric新增一台Nginx监控
hosts: ["http://xxx","http://xxx"]

一台Metricbeat就能监控能通的Nginx

metricbeat开启nginx监测模块

Metricbeat采集Redis信息Grafana展示

安装redis
  yum install redis -y

配置redis
  设置绑定地址：0.0.0.0
  设置密码：sjgpwd

Metricbeat支持redis
  metricbeat modules enable redis

监控redis每秒连接数
  received persecond _value / 60
  Derivative
  grouby: service.address
  alias: {{service.address.keyword}} received persecond

模拟数据
  while true; do redis-cli -a sjgpwd set a b; redis-cli -a sjgpwd get a; sleep 1; done

metricbeat还支持收集很多模块，见以下目录
/etc/metricbeat/modules.d/

redis

Metricbeat采集Mysql监控数据

Metricbeat采集Mysql监控数据
安装mysql服务器准备
yum install mariadb-server -y
systemctl restart mariadb
mysql_secure_installation

监控权限开启
grant usage on *.* to 'monitor'@'192.168.%' identified by 'sjgpwd';
flush privileges;

测试能否正常登录
mysql -h 192.168.238.92 -umonitor -psjgpwd -A

开启监控mysql模块
metricbeat modules enable mysql

配置mysql监控
  hosts: ["monitor:sjgpwd@tcp(xxx:3306)/"]

监控mysql delete persecond
  delete persecond _value / 60
  Derivative
  grouby: service.address
  alias: {{service.address.keyword}} delete persecond

模拟数据，观察grafana
use mysql;
create table test (id int);
mysql -psjgpwd -A -e "use mysql; delete from test"

mysql

Metricbeat根据不同类型定义不同索引

metricbeat采集
  metricbeat modules enable system
  metricbeat modules enable redis
  metricbeat modules enable nginx
  metricbeat modules enable mysql

Logstash区分system还是redis 还是其它
output {
  if [service][type] == "mysql" {
    elasticsearch {
      hosts => ["http://192.168.238.90:9200", "http://192.168.238.92:9200"]
      user => "elastic"
      password => "sjgpwd"
      index => "sjgmysql-%{+YYYY.MM.dd}"
    }
  }
  else if [service][type] == "redis" {
    elasticsearch {
      hosts => ["http://192.168.238.90:9200", "http://192.168.238.92:9200"]
      user => "elastic"
      password => "sjgpwd"
      index => "sjgredis-%{+YYYY.MM.dd}"
    }
  }
  else {
    elasticsearch {
      hosts => ["http://192.168.238.90:9200", "http://192.168.238.92:9200"]
      user => "elastic"
      password => "sjgpwd"
      index => "sjgother-%{+YYYY.MM.dd}"
    }
  }
}

logstash配置

Kibana页面上ES索引监控的开启

Kibana监控的开启
  开启kibana监控
  使用metricbeat

启动metricbeat
  metricbeat modules enable elasticsearch-xpack
  metricbeat modules disable 所有

配置文件elasticsearch-xpack.yml
- module: elasticsearch
  metricsets:
    - ccr
    - cluster_stats
    - enrich
    - index
    - index_recovery
    - index_summary
    - ml_job
    - node_stats
    - shard
  period: 10s
  hosts: ["http://192.168.238.90:9200", "http://192.168.238.92:9200"]
  username: "elastic"
  password: "sjgpwd"
  xpack.enabled: true

metricbeat配置：/etc/metricbeat/metricbeat.yml
metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression
setup.kibana:

output.elasticsearch:
  hosts: ["192.168.238.90:9200", "192.168.238.92:9200"]
  username: "elastic"
  password: "sjgpwd"

Python测试监控是否正常
import time
import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch(['http://elastic:sjgpwd@192.168.238.90:9200', 'http://elastic:sjgpwd@192.168.238.92:9200'])
for i in range(10000):
  curtime=datetime.datetime.utcnow().isoformat()
  body = {"name": "sjg{0}".format(i), "@timestamp": curtime, "sjgcount": i}
  es.index(index='sjg', body=body)
  time.sleep(1)
  print('insert {0}'.format(i))

kibana监测es