ELK批量删除索引

一、存在问题

用了一段时间elk发现如果索引长时间不删除，elk会越来越慢，重启elasticsearch服务器节点之前同步时间也会很长

二、解决方法（定期删除索引）

1.在elasticsearch节点上使用curl -XGET 'http://192.168.X.XX:9200/_cat/shards'查看索引

[root@--x-x scripts]# curl -XGET 'http://192.168.x.x:9200/_cat/shards' | more
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
                      --:--:-- --:--:-- --:--:--     0freetrip-2019.01.    r STARTED     59kb 192.168.x.x node3-192.168.x.x
freetrip-2019.01.                 p STARTED         59kb 192.168.x.x node1-192.168.x.x
freetrip-2019.01.                 p STARTED      .7kb 192.168.x.x node3-192.168.x.x
freetrip-2019.01.                 r STARTED      .7kb 192.168.x.x node1-192.168.x.x
freetrip-2019.01.                 p STARTED       .5kb 192.168.x.x node2-192.168.x.x
freetrip-2019.01.                 r STARTED       .5kb 192.168.x.x node1-192.168.x.x
freetrip-2019.01.                 r STARTED       .5kb 192.168.x.x node2-192.168.x.x
freetrip-2019.01.                 p STARTED       .5kb 192.168.x.x node3-192.168.x.x
freetrip-2019.01.                 p STARTED       .9kb 192.168.x.x node2-192.168.x.x
freetrip-2019.01.                 r STARTED       .9kb 192.168.x.x node3-192.168.x.x

2.将过滤出来要删除的索引存到临时文件夹里面

[root@--x-x ~]# curl -XGET 'http://192.168.x.x:9200/_cat/shards' | grep 192.168.x.x | grep  | awk '{print $1}' |uniq > elk-index.tmp
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
                142k       --:--:-- --:--:-- --:--:--  142k
[root@--- ~]# cat elk-index.tmp
otter-canal--2018.12.
logstash-nginx-chm-accesslog-2018.11.
logstash-nginx-oms-accesslog-2018.10.
laravel-chm248-online-2018.12.
otter-canal--2018.11.
logstash-nginx-oms-accesslog-2018.11.
otter-communication--2018.11.
otter-communication--2018.12.
otter-node--2018.12.
logstash-nginx-chvisa-accesslog-2018.10.
wanmei219-online-2018.10.
logstash-nginx-chm-accesslog-2018.11.
otter-canal--2018.11.
otter-canal--2018.11.
logstash-nginx-oms-accesslog-2018.11.
laravel-chm248-online-2018.11.
logstash-nginx-oms-accesslog-2018.12.
otter-manager--2018.12.
logstash-nginx-chvisa-accesslog-2018.11.
wanmei219-online-2018.12.
logstash-nginx-chvisa-accesslog-2018.11.
otter-communication--2018.11.
otter-communication--2018.11.
laravel-chm248-online-2018.11.
laravel-chm248-online-2018.10.
logstash-nginx-chm-accesslog-2018.12.
wanmei219-online-2018.11.
logstash-nginx-chvisa-accesslog-2018.12.
wanmei219-online-2018.11.
logstash-nginx-chm-accesslog-2018.10.

3. 删除过滤出来的索引文件

for i in `cat elk-index.tmp`
do
      curl -XDELETE  http://192.168.3.163:9200/$i
done

4. 使用脚本加定时任务，每一天删除前三天的索引

[root@--x-x ~]# cat /home/scripts/del_elasticseatch_index.sh
#!/bin/bash
curl -XGET 'http://192.168.x.x:9200/_cat/shards' |grep 192.168.x.x | awk '{print $1}' |grep `date -d "5 days ago" +%Y.%m.%d` | uniq > /tmp/index_name.tmp

for index_name in `cat /tmp/index_name.tmp`
do
   echo $index_name
    curl -XDELETE  http://192.168.x.x:9200/$index_name
    echo "${index_name} delete success" >> /home/scripts/del_elasticseatch_index.log
done

5. 定时任务

[root@--x-x ~]# crontab -l
  * * * bash /home/scripts/del_elasticseatch_index.sh