logstash (?m) 经典例子

在和 codec/multiline 搭配使用的时候，需要注意一个问题，grok 正则和普通正则一样，默认是不支持匹配回车换行的。就像你需要 =~ //m 一样也需要单独指定，具体写法是在表达式开始位置加 (?m) 标记。

\s  空格,和 [\n\t\r\f] 语法一样 

(\s*\S+\s*).* 匹配0个或者多个前导字符

简单demo:

 SELECT t.*  FROM
 	    (
 			SELECT
 			t1.sn AS clientSn,
 			t1.userNick,
 			t1.mobilePhone,
 			t3.personName,
 			t2.availableBalance,
 			(SELECT IFNULL(SUM(amount) , 0) FROM ClientRechargeOrder t WHERE t.clientSn= t1.sn AND t.status ='2') AS rechargeAmount,
 			(SELECT IFNULL(SUM(amount) , 0) FROM ClientWithDrawOrder t WHERE t.clientSn= t1.sn AND t.status IN ('1','2','3','4') ) AS withdrawAmount,
 			( (SELECT IFNULL(SUM(capitalBalance) , 0) FROM ProductRepayment t WHERE t.clientSn= t1.sn AND t.status= '1')
 			  +
 			  (SELECT IFNULL(SUM(capitalBalance) , 0) FROM VirtualProductOrder t WHERE t.clientSn= t1.sn AND t.status= '1')
 			) AS investAmount,
 			( (SELECT IFNULL(SUM(yieldBalance) , 0) FROM ProductRepayment t WHERE t.clientSn= t1.sn AND t.status= '2')
 			  +
 			  (SELECT IFNULL(SUM(yieldBalance) , 0) FROM VirtualProductOrder t WHERE t.clientSn= t1.sn AND t.status= '2')
 			) AS yieldAmount,
 			(SELECT IFNULL(SUM(t0.amount) , 0) FROM ClientCoupon t,Coupon t0 WHERE t.clientSn= t1.sn AND t.status = '2' AND t.couponSn = t0.sn AND t0.type IN (1,2)) AS cashCouponAmount
 			FROM  Client t1 , ClientAssetInfo t2 , ClientPersonalInfo t999
 			WHERE t1.sn = t2.clientSn AND t1.sn = t3.clientSn
 	    ) t  WHERE (t.rechargeAmount + t.yieldAmount + t.cashCouponAmount - t.withdrawAmount - t.investAmount - t.availableBalance) != 0;

正则表达式；
\s*(?<query>(\s*\S+\s*).*)\s*

匹配结果:
{
  "query": [
    [
      "SELECT t.*  FROM"
    ]
  ]
}

////////////////////////////////////

正则表达式：
(?m)\s*(?<query>(\s*\S+\s*).*)\s*

匹配结果:

{
  "query": [
    [
      "SELECT t.*  FROM\n \t    (\n \t\t\tSELECT \n \t\t\tt1.sn AS clientSn,\n \t\t\tt1.userNick,\n \t\t\tt1.mobilePhone,\n \t\t\tt3.personName,\n \t\t\tt2.availableBalance,\n \t\t\t(SELECT IFNULL(SUM(amount) , 0) FROM ClientRechargeOrder t WHERE t.clientSn= t1.sn AND t.status ='2') AS rechargeAmount,\n \t\t\t(SELECT IFNULL(SUM(amount) , 0) FROM ClientWithDrawOrder t WHERE t.clientSn= t1.sn AND t.status IN ('1','2','3','4') ) AS withdrawAmount,\n \t\t\t( (SELECT IFNULL(SUM(capitalBalance) , 0) FROM ProductRepayment t WHERE t.clientSn= t1.sn AND t.status= '1') \n \t\t\t  + \n \t\t\t  (SELECT IFNULL(SUM(capitalBalance) , 0) FROM VirtualProductOrder t WHERE t.clientSn= t1.sn AND t.status= '1')  \n \t\t\t) AS investAmount,\n \t\t\t( (SELECT IFNULL(SUM(yieldBalance) , 0) FROM ProductRepayment t WHERE t.clientSn= t1.sn AND t.status= '2') \n \t\t\t  + \n \t\t\t  (SELECT IFNULL(SUM(yieldBalance) , 0) FROM VirtualProductOrder t WHERE t.clientSn= t1.sn AND t.status= '2')  \n \t\t\t) AS yieldAmount,\n \t\t\t(SELECT IFNULL(SUM(t0.amount) , 0) FROM ClientCoupon t,Coupon t0 WHERE t.clientSn= t1.sn AND t.status = '2' AND t.couponSn = t0.sn AND t0.type IN (1,2)) AS cashCouponAmount\n \t\t\tFROM  Client t1 , ClientAssetInfo t2 , ClientPersonalInfo t999\n \t\t\tWHERE t1.sn = t2.clientSn AND t1.sn = t3.clientSn\n \t    ) t  WHERE (t.rechargeAmount + t.yieldAmount + t.cashCouponAmount - t.withdrawAmount - t.investAmount - t.availableBalance) != 0;\n\n\n"
    ]
  ]
}

继续测试；

表达式；

\s*(?<query>(\S+\n*).*)\s*

输出:

{
  "query": [
    [
      "SELECT t.*  FROM"
    ]
  ]
}

正则:
(?m)\s*(?<query>(\S+\n*).*)\s*

Grok Debugger

    Debugger
    Discover
    Patterns

Add custom patterns Keep Empty Captures Named Captures Only Singles Autocomplete  

{
  "query": [
    [
      "SELECT t.*  FROM\n \t    (\n \t\t\tSELECT \n \t\t\tt1.sn AS clientSn,\n \t\t\tt1.userNick,\n \t\t\tt1.mobilePhone,\n \t\t\tt3.personName,\n \t\t\tt2.availableBalance,\n \t\t\t(SELECT IFNULL(SUM(amount) , 0) FROM ClientRechargeOrder t WHERE t.clientSn= t1.sn AND t.status ='2') AS rechargeAmount,\n \t\t\t(SELECT IFNULL(SUM(amount) , 0) FROM ClientWithDrawOrder t WHERE t.clientSn= t1.sn AND t.status IN ('1','2','3','4') ) AS withdrawAmount,\n \t\t\t( (SELECT IFNULL(SUM(capitalBalance) , 0) FROM ProductRepayment t WHERE t.clientSn= t1.sn AND t.status= '1') \n \t\t\t  + \n \t\t\t  (SELECT IFNULL(SUM(capitalBalance) , 0) FROM VirtualProductOrder t WHERE t.clientSn= t1.sn AND t.status= '1')  \n \t\t\t) AS investAmount,\n \t\t\t( (SELECT IFNULL(SUM(yieldBalance) , 0) FROM ProductRepayment t WHERE t.clientSn= t1.sn AND t.status= '2') \n \t\t\t  + \n \t\t\t  (SELECT IFNULL(SUM(yieldBalance) , 0) FROM VirtualProductOrder t WHERE t.clientSn= t1.sn AND t.status= '2')  \n \t\t\t) AS yieldAmount,\n \t\t\t(SELECT IFNULL(SUM(t0.amount) , 0) FROM ClientCoupon t,Coupon t0 WHERE t.clientSn= t1.sn AND t.status = '2' AND t.couponSn = t0.sn AND t0.type IN (1,2)) AS cashCouponAmount\n \t\t\tFROM  Client t1 , ClientAssetInfo t2 , ClientPersonalInfo t999\n \t\t\tWHERE t1.sn = t2.clientSn AND t1.sn = t3.clientSn\n \t    ) t  WHERE (t.rechargeAmount + t.yieldAmount + t.cashCouponAmount - t.withdrawAmount - t.investAmount - t.availableBalance) != 0;\n\n\n"
    ]
  ]
}

“I grok in fullness.” Robert A. Heinlein, Stranger in a Strange Land