ELK 学习笔记之 Logstash之filter配置

Logstash之filter：

json filter：

input{

stdin{

}

}

filter{

json{

source => "message"

}

}

output{

stdout{

codec => json

}

输入：

{"name": "CSL", "age": 20}

输出：

Grok filter:

pattern：

https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns

创建一个测试log：

[sky@hadoop1 bin]$ cat spark-test-log.log

05/30/17 17:13:24 INFO StartingSparkmasteratspark

05/30/17 17:13:24 INFO RunningSparkversion1

05/30/17 17:13:25 INFO jetty

创建conf：

input{

file{

path => "/usr/local/logstash-5.6.1/bin/spark-test-log.log"

type => "sparkfile"

start_position => "beginning"

}

}

filter{

grok{

match => ["message", "%{DATE:date} %{TIME:time} %{LOGLEVEL:loglevel} %{WORD:word}"]

}

}

output{

stdout{

codec => rubydebug

}

}

运行结果：

自定义正则表达式：

[sky@hadoop1 patterns]$ cat selfpattern

SKYTIME (?!<[0-9])%{HOUR}:%{MINUTE}(?::%{SECOND})(?![0-9])

修改conf:

input{

file{

path => "/usr/local/logstash-5.6.1/bin/spark-test-log.log"

type => "sparkfile"

start_position => "beginning"

}

}

filter{

grok{

patterns_dir => '/usr/local/logstash-5.6.1/patterns/selfpattern'

match => ["message", "%{DATE:date} %{SKYTIME:time} %{LOGLEVEL:loglevel} %{WORD:word}"]

}

}

output{

stdout{

codec => rubydebug

}

}

输出结果：

定义多个match：使用，分隔。

测试正则表达式网址：

https://grokdebug.herokuapp.com/