编写一个简单的基于jmespath 的prometheus exporter

目的很简单，因为系统好多监控指标是通过json 暴露的，并不是标准的prometheus metrics 格式，处理方法
实际上很简单，我们可以基于jsonpath 解析json数据，转换为prometheus metrics 但是感觉查询能力一般，个人
还是比较喜欢jmespath，目前已经有一个开源的基于jsonpath 开发的exporter prometheus-jsonpath-exporter,基于python 开发，所以使用jmespath 直接替换jsonpath 实现更加方便的exporter

项目结构

代码很简单，基于jsonpath 的exporter开发， 替换jsonpath library 为 jmespath

• 整体结构

├── Dockerfile
├── LICENSE
├── README.md
├── app
│ ├── exporter.py
│ └── requirements.txt
└── example
    └── config.yml

• app/requirements.txt

pytz
prometheus_client
jmespath
pyYAML

• app/exporter.py

#!/usr/bin/python

import json
import time
import urllib2
from prometheus_client import start_http_server
from prometheus_client.core import GaugeMetricFamily, REGISTRY
import argparse
import yaml
import jmespath
import logging
DEFAULT_PORT=9158
DEFAULT_LOG_LEVEL='info'

class JsonPathCollector(object):
  def __init__(self, config):
    self._config = config

  def collect(self):
    config = self._config
    result = json.loads(urllib2.urlopen(config['json_data_url'], timeout=10).read())
    for metric_config in config['metrics']:
      metric_name = "{}_{}".format(config['metric_name_prefix'], metric_config['name'])
      metric_description = metric_config.get('description', '')
      metric_path = metric_config['path']
      // jmeshpath metrics 处理
      value =jmespath.search(metric_path,result)
      logging.debug("metric_name: {}, value for '{}' : {}".format(metric_name, metric_path, value))
      metric = GaugeMetricFamily(metric_name, metric_description, value=value)
      yield metric

if __name__ == "__main__":
  parser = argparse.ArgumentParser(description='Expose metrics bu jsonpath for configured url')
  parser.add_argument('config_file_path', help='Path of the config file')
  args = parser.parse_args()
  with open(args.config_file_path) as config_file:
    config = yaml.load(config_file)
    log_level = config.get('log_level', DEFAULT_LOG_LEVEL)
    logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.getLevelName(log_level.upper()))
    exporter_port = config.get('exporter_port', DEFAULT_PORT)
    logging.debug("Config %s", config)
    logging.info('Starting server on port %s', exporter_port)
    start_http_server(exporter_port)
    REGISTRY.register(JsonPathCollector(config))
  while True: time.sleep(1)

• dockerfile

FROM python:2.7.13-alpine
COPY app /opt/prometheus-jmespath-exporter
RUN pip install -r /opt/prometheus-jmespath-exporter/requirements.txt
EXPOSE 9158
ENTRYPOINT ["python", "/opt/prometheus-jmespath-exporter/exporter.py"]

测试使用

• 构建容器镜像

docker build -t dalongrong/promethues-jmespath-exporter .

• 使用

  一个简单的例子，使用了goaccess 进行nginx 日志的处理，包含了geoip 的处理,具体json api 格式参考goaccess-geoip-docker-compose-demo

version: "3"
services:
  metrics:
   image: dalongrong/promethues-jmespath-exporter
   volumes:
   - "./conf/config.yaml:/etc/prometheus-jsonpath-exporter/config.yml"
   ports:
   - "9158:9158"
   command: /etc/prometheus-jsonpath-exporter/config.yml

• confi.yaml

exporter_port: 9158 # Port on which prometheus can call this exporter to get metrics
log_level: info
json_data_url: http://ipaddress||docker-compose-servicename/app.json # Url to get json data used for fetching metric values
metric_name_prefix: go_access # All metric names will be prefixed with this value
metrics:
- name: total_request_ip # Final metric name will be go_access_total_request
  description: Number of total request ip address
  path: length(hosts.data[*].data)

• 效果
  

说明

主要还是基于别人写好的一个exporter，只是做了简单的修改，使用jmespath进行metrics 指标的转换，主要是jmespath 能力强大，同时docker 镜像
也已经push docker 官方仓库了promethues-jmespath-exporter

参考资料

https://github.com/rongfengliang/goaccess-geoip-docker-compose-demo
https://github.com/project-sunbird/prometheus-jsonpath-exporter
https://github.com/rongfengliang/promethues-jmespath-exporter
https://www.cnblogs.com/rongfengliang/p/10688965.html