Zabbix监控disk performance

概述

zabbix获取/sys里面的磁盘信息并分析来监控disk performance

sysfs是Linux内核中设计较新的一种虚拟的基于内存的文件系统，它的作用与 proc 有些类似（默认挂载在/sys）

sysfs is a pseudo file system provided by the Linux kernel that exports information about various kernel subsystems, hardware devices, and associated device drives from the kernel's device model to user space through virtual files. In addition to providing information about various devices and kernel subsystems, exported virtual files are also used for their configuring. （维基百科的解释）

原理

查看磁盘，分区状态值的统计信息

/sys/block/<dev>/stat

[root@test ~]# cat /sys/class/block/sda/stat
                                       

Field   Name            units         description
-----   ----            -----         -----------
       read I/Os       requests      number of read I/Os processed
       read merges     requests      number of read I/Os merged with in-queue I/O
       read sectors    sectors       number of sectors read
       read ticks      milliseconds  total wait time for read requests
       write I/Os      requests      number of write I/Os processed
       write merges    requests      number of write I/Os merged with in-queue I/O
       write sectors   sectors       number of sectors written
       write ticks     milliseconds  total wait time for write requests
       in_flight       requests      number of I/Os currently in flight
      io_ticks        milliseconds  total time this block device has been active
      time_in_queue   milliseconds  total wait time for all requests

merge：读取/写请求有多少被Merge了(当系统调用需要读取/写数据的时候, VFS将请求发到各个分区, 如果分区发现不同的读取/写请求的是相同Block的数据,

FS会将这个请求合并Merge)，所以磁盘读写吞吐量（throughput）= 读写扇区（field3,7）次数 * 扇区大小 / 时间s，而不是读写merge后的iops（field1,5）

ps: 除了Field 9其他的Field都是自上次系统启动后的累加值

配置

定义zabbix监控item的配置文件

# disk_discovery
UserParameter=custom.vfs.discover_disks,/m2odata/server/zabbix-agent/scripts/lld-disks.py
# disk_stats
UserParameter=custom.vfs.dev.read.ops[*],awk '{print $$1}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.read.merged[*],awk '{print $$2}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.read.sectors[*],awk '{print $$3}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.read.ms[*],awk '{print $$4}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.write.ops[*],awk '{print $$5}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.write.merged[*],awk '{print $$6}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.write.sectors[*],awk '{print $$7}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.write.ms[*],awk '{print $$8}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.io.active[*],awk '{print $$9}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.io.ms[*],awk '{print $$10}' /sys/class/block/$/stat
UserParameter=custom.vfs.dev.weight.io.ms[*],awk '{print $$11}' /sys/class/block/$/stat

磁盘自动发现规则

lld-disk.py

#!/usr/bin/env python

import os
import re
import json

def Devices(diskdir, skippable):
    raw_devices = (device for device in os.listdir(diskdir) if not any(ignore in device for ignore in skippable))
    devices = (device for device in raw_devices if re.match(r'^\w{3}$', device))  # 保留整块磁盘 去掉分区, such as remove sda1 sdb2
    data = [{"{#DEVICENAME}": device} for device in devices]
    print(json.dumps({"data": data}, indent=4))

if __name__ == "__main__":
    # Iterate over all block devices, but ignore them if they are in the skippable set
    diskdir = "/sys/class/block"
    skippable = ("sr", "loop", "ram", "dm")
    Devices(diskdir, skippable)

重启zabbix_agentd使配置文件生效

服务器端  # 创建监控disk模板(Template Disk Performance)

Discovery rule

Item prototypes

创建Graph prototype（iops,throughput）

查看graph

zabbix后台 Monitoring->Graphs->对应的graph

参考：

/sys/block/<dev>/stat:　　https://www.kernel.org/doc/Documentation/block/stat.txt

/proc/diskstats:　　https://www.kernel.org/doc/Documentation/ABI/testing/procfs-diskstats

iostat:　　https://www.kernel.org/doc/Documentation/iostats.txt

github: 　https://github.com/grundic/zabbix-disk-performance