linux 下jq的使用

安装：

yum install jq -y

文档：

https://stedolan.github.io/jq/manual/

更多:

https://blog.csdn.net/Cheat1173010256/article/details/118230562

数据源

333.json

[{
    "name": "站长工具",
    "url": "http://tool.chinaz.com",
    "address": {
      "city": "厦门",
      "country": "中国"
    },
    "arrayBrowser": [{
        "name": "Google",
        "url": "http://www.google.com"
      },
      {
        "name": "Baidu",
        "url": "http://www.baidu.com"
      }
    ]
  },
  {
    "name": "站长之家",
    "url": "http://tool.zzhome.com",
    "address": {
      "city": "大连",
      "country": "中国"
    },
    "arrayBrowser": [{
        "name": "360",
        "url": "http://www.so.com"
      },
      {
        "name": "bing",
        "url": "http://www.bing.com"
      }
    ],
    "127.0.0.1": {
      "error": 200,
      "msg": "ok"
    }
  }
]

例子基础

haima@haima-PC:~/Desktop$ cat 333.json |jq '.'
[
  {
    "name": "站长工具",
    "url": "http://tool.chinaz.com",
    "address": {
      "city": "厦门",
      "country": "中国"
    },
    "arrayBrowser": [
      {
        "name": "Google",
        "url": "http://www.google.com"
      },
      {
        "name": "Baidu",
        "url": "http://www.baidu.com"
      }
    ]
  },
  {
    "name": "站长之家",
    "url": "http://tool.zzhome.com",
    "address": {
      "city": "大连",
      "country": "中国"
    },
    "arrayBrowser": [
      {
        "name": "360",
        "url": "http://www.so.com"
      },
      {
        "name": "bing",
        "url": "http://www.bing.com"
      }
    ],
    "127.0.0.1": {
      "error": 200,
      "msg": "ok"
    }
  }
]

管道线 |

jq支持管道线 |，它如同linux命令中的管道线——把前面命令的输出当作是后面命令的输入。如下命令把.[0]作为{…}的输入，进而访问嵌套的属性，如.name和.address.city。

观察如下几个命令，通过改变|前后的输入和输出来达到不同的效果：

cat json.txt | jq '.[0] | {name:.name,city:.address.city}'

{
  "name": "站长工具",
  "city": "厦门"
}

cat json.txt | jq '.[0] | {name:.arrayBrowser[1].name,city:.address.city}'

{
  "name": "Baidu",
  "city": "厦门"
}

cat json.txt | jq ".[] | {name:.arrayBrowser[1].name,city:.address.city}"

{
  "name": "Baidu",
  "city": "厦门"
}
{
  "name": "bing",
  "city": "大连"
}

[]
如果希望把jq的输出当作一个数组，可以在前后加上[]：

cat json.txt | jq "[.[] | {name:.arrayBrowser[1].name,city:.address.city}]"
[
  {
    "name": "Baidu",
    "city": "厦门"
  },
  {
    "name": "bing",
    "city": "大连"
  }
]

自定义key
在{}中，冒号前面的名字是映射的名称，你可以任意修改，如：

```sh
cat json.txt | jq "[.[] | {name_001:.arrayBrowser[1].name,city_002:.address.city}]"

[
  {
    "name_001": "Baidu",
    "city_002": "厦门"
  },
  {
    "name_001": "bing",
    "city_002": "大连"
  }
]

原文链接：https://blog.csdn.net/qq_26502245/article/details/100191694

取0号单元下的元素

haima@haima-PC:~/Desktop$ cat 333.json |jq '.[0].name'
"站长工具"
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[0].address'
{
  "city": "厦门",
  "country": "中国"
}
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[0].address.city'
"厦门"
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[0].address["city"]'
"厦门"
haima@haima-PC:~/Desktop$ cat 333.json |jq ".[0].address[\"city\"]"
"厦门"
haima@haima-PC:~/Desktop$ cat 333.json |jq '.[1]["127.0.0.1"]'
{
  "error": 200,
  "msg": "ok"
}
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0]" | jq 'keys' #获取0号单元的所有key
[
  "address",
  "arrayBrowser",
  "name",
  "url"
]
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0].address" # 获取0号单元的address
{
  "city": "厦门",
  "country": "中国"
}
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0].address" | jq 'keys' #获取0号单元的address里的所有 key(嵌套提取)
[
  "city",
  "country"
]
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0]" | jq 'has("name")' #判断是否有某个 key
true
haima@haima-PC:~/Desktop$ cat 333.json | jq ".[0]" | jq 'has("test")' #判断是否有某个 key
false

压缩json

haima@haima-PC:~/Desktop$ cat 333.json | jq -c .
[{"name":"站长工具","url":"http://tool.chinaz.com","address":{"city":"厦门","country":"中国"},"arrayBrowser":[{"name":"Google","url":"http://www.google.com"},{"name":"Baidu","url":"http://www.baidu.com"}]},{"name":"站长之家","url":"http://tool.zzhome.com","address":{"city":"大连","country":"中国"},"arrayBrowser":[{"name":"360","url":"http://www.so.com"},{"name":"bing","url":"http://www.bing.com"}],"127.0.0.1":{"error":200,"msg":"ok"}}]

例子二:

数据源 22.json

{"ip":"34.120.75.75"}
{"ip":"35.193.30.131"}
{"ip":"35.227.228.135"}
{"ip":"130.211.21.179"}
{"ip":"35.192.215.111"}
{"ip":"35.214.27.221"}
{"ip":"34.120.136.4"}
{"ip":"35.209.185.188"}
{"ip":"34.121.213.115"}
{"ip":"34.98.116.47"}

取出ip

$ cat 22.json |jq '.ip'
"34.120.75.75"
"35.193.30.131"
"35.227.228.135"
"130.211.21.179"
"35.192.215.111"
"35.214.27.221"
"34.120.136.4"
"35.209.185.188"
"34.121.213.115"
"34.98.116.47"

取出ip并去除双引号 (需要安装jq)

$ cat 22.json |jq '.ip' | sed 's/\"//g'
34.120.75.75
35.193.30.131
35.227.228.135
130.211.21.179
35.192.215.111
35.214.27.221
34.120.136.4
35.209.185.188
34.121.213.115
34.98.116.47

排序去重后,取出ip并去除双引号 (需要安装jq)

$  cat 22.json |sort | uniq  | sort -n | jq '.ip' | sed 's/\"//g'
34.120.241.13
34.125.140.101
34.126.67.33
34.65.22.11
34.77.137.149
34.85.82.186
34.86.160.16
34.90.193.87
34.92.44.105
34.93.161.137
34.94.248.178

解析：

s 为替换

以g结尾表示的是:全局性,意即”替代文本取代正则表达式中每一个匹配的”

直接改变的话用i：

sed i 's/\"//g'  11.txt

另外输出的话：

sed 's/\"//g'  aa.txt > 11.clean.txt

例子:

取出port和value ,组装json格式

{"took":648,"timed_out":false,"_shards":{"total":50,"successful":50,"skipped":0,"failed":0},"hits":{"total":9893731,"max_score":1.0,"hits":[{"_index":"fofapro_service","_type":"order","_id":"1.169.157.54:80","_score":1.0,"_source":{"port":80}}]},"aggregations":{"distinct_ips":{"value":9883243}}}
{"took":648,"timed_out":false,"_shards":{"total":50,"successful":50,"skipped":0,"failed":0},"hits":{"total":9893731,"max_score":1.0,"hits":[{"_index":"fofapro_service","_type":"order","_id":"1.169.157.54:80","_score":1.0,"_source":{"port":81}}]},"aggregations":{"distinct_ips":{"value":9883243}}}

命令:

$ cat es_port.json | jq '{port:.hits.hits[0]._source.port , count:.aggregations.distinct_ips.value}' -c | awk -F'port":' '{print $2}' | awk -F',"count":' '{print "{\"" $1"\":\""$2}' | sed 's/}/"}/g'
{"80":"9883243"}
{"81":"9883243"}

-c ：一行显示

4 JSON parse array

   cat json_raw.txt | jq '.employees[1].name'
   "Laura"

5 内建函数

jq还有一些内建函数如 key，has

key是用来获取JSON中的key元素的：

cat json_raw.txt | jq 'keys'
[
  "employees",
  "location",
  "name"
]

has是用来是判断是否存在某个key：

cat json_raw.txt | jq 'has("name")'
true

cat json_raw.txt | jq 'has("noexisted")'
false