【笔记】VictoriaMetrics中，对大量的pull模式的targets进行分片

作者:张富春(ahfuzhang)，转载时请注明作者和引用链接，谢谢！

• cnblogs博客
• zhihu
• Github
• 公众号:一本正经的瞎扯
  
  

---

配置的方法请看这里：https://docs.victoriametrics.com/vmagent.html#scraping-big-number-of-targets

具体的配置文档如下：

  -promscrape.cluster.memberNum string
     The number of number in the cluster of scrapers. It must be an unique value in the range 0 ... promscrape.cluster.membersCount-1 across scrapers in the cluster. Can be specified as pod name of Kubernetes StatefulSet - pod-name-Num, where Num is a numeric part of pod name (default "0")
  -promscrape.cluster.membersCount int
     The number of members in a cluster of scrapers. Each member must have an unique -promscrape.cluster.memberNum in the range 0 ... promscrape.cluster.membersCount-1 . Each member then scrapes roughly 1/N of all the targets. By default cluster scraping is disabled, i.e. a single scraper scrapes all the targets
  -promscrape.cluster.name string
     Optional name of the cluster. If multiple vmagent clusters scrape the same targets, then each cluster must have unique name in order to properly de-duplicate samples received from these clusters. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679
  -promscrape.cluster.replicationFactor int
     The number of members in the cluster, which scrape the same targets. If the replication factor is greater than 1, then the deduplication must be enabled at remote storage side. See https://docs.victoriametrics.com/#deduplication (default 1)
  -promscrape.config string
        Optional path to Prometheus config file with 'scrape_configs' section containing targets to scrape. The path can point to local file and to http url. See https://docs.victoriametrics.com/#how-to-scrape-prometheus-exporters-such-as-node-exporter for details

简要来说就是：

• -promscrape.cluster.membersCount=${vm-agent的总数}，例如我启动了10个vm-agent，这里写10
• -promscrape.cluster.memberNum=${0~count-1} , 说明当前是第N个。
• -promscrape.config=prometheus.yml，在配置文件中通过static, file_sd, httpsd等来配置target的位置
  • 当然，通过httpsd，可以由服务器端来做分片。这里不讨论这种模式
  • -promscrape.configCheckInterval=60s来配置配置文件重新加载的周期
• -promscrape.cluster.replicationFactor=1，复制因子默认是1.
  • 为了vm-agent这一层的高可用，可以配置为2或者更高。
  • 配置了复制因子后，vm-storage一定要配置去重的周期：-dedup.minScrapeInterval=15s。去重周期必须与vm-agent的抓取周期一致。

最后：-promscrape.cluster.name=guangzhou/shenzhen，这个参数看起来是为了分可用区域来配置vm-agent群集的名字。看起来还没发布。

关于这个参数的解释，请见：https://github.com/VictoriaMetrics/VictoriaMetrics/issues/2679