假如现有数据：

{ "@timestamp": "2018-10-13T21:55:58+08:00", "remote_addr": "100.120.34.3", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.076, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.3" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.076 }
{ "@timestamp": "2018-10-13T21:56:06+08:00", "remote_addr": "100.120.34.101", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.076, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.101" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.076 }
{ "@timestamp": "2018-10-13T21:56:08+08:00", "remote_addr": "100.120.34.29", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.075, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.29" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.075 }
{ "@timestamp": "2018-10-13T21:56:10+08:00", "remote_addr": "100.120.34.75", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=D166459D-E823-4847-9094-6F4BF90625B2&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.078, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.75" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.078 }
{ "@timestamp": "2018-10-13T21:56:18+08:00", "remote_addr": "100.120.34.39", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=08C65C3B-EED2-4A65-B0C1-67FC7FB78E18&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.082, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.39" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.082 }
{ "@timestamp": "2018-10-13T21:56:31+08:00", "remote_addr": "100.120.34.68", "referer": "-", "request": "GET /api/gourd/activeupload?idfa=D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.079, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.68" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.079 }

临时保存为tmp.log

awk -F 'idfa=' '{print $2}' tmp.log

出现如下结果：

58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.076, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.3" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.076 }
E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.076, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.101" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.076 }
D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206&source=rehulu HTTP/1.1", "status": 200, "request_time": 0.075, "cookie":"-","host":"cms.369wan.com","bytes": 48, "agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0", "proxy_x_forwarded": "139.129.97.187, 100.120.34.29" "upstr_addr": "127.0.0.1:9000","upstr_host": "-","ups_resp_time": 0.075 }

执行

awk -F 'idfa=' '{print $2}' tmp.log | awk -F '&source=' '{print $1}'

出现如下结果：
58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206
E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206
D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206

执行

awk -F 'idfa=' '{print $2}' tmp.log | awk -F '&source=' '{print $1}'  | sort  (这一步将结果相同的放在一起)

出现如下结果：
58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206
E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206
D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206

执行

awk -F 'idfa=' '{print $2}' cms_\(2\).log | awk -F '&source=' '{print $1}' |sort| uniq -c
出现最终结果(次数 和 各字符串)：
1 58237FA9-A1B3-4202-B5F3-9536983119E5&appid=1410137206
1 E9D7F87A-9042-46B4-82E8-E5F64B74466B&appid=1410137206
1 D5B924F3-7D25-4B52-BAE9-3270B08EA32D&appid=1410137206

centos下对文件某些特定字符串分组统计出现次数的更多相关文章

  1. centos下修改文件后如何保存退出

    centos下修改文件后如何保存退出 保存命令 按ESC键 跳到命令模式,然后: :w 保存文件但不退出vi :w file 将修改另外保存到file中,不退出vi :w! 强制保存,不推出vi :w ...

  2. 在文件夹下所有文件中查找字符串(linux/windows)

    在linux下可以用 grep "String" filename.txt#字符串 文件名grep -r "String" /home/#递归查找目录下所有文件 ...

  3. MyEclipse如何查找指定工程下所有或指定文件中特定字符串并且可进行批量替换

    查找操作步骤:(1)在myEclipse里菜单选择-Search-Search(快捷键:ctrl+h);(2)在弹出对话框中选File Search选项,然后在第一个文本框中输入“要查找的字符串”(为 ...

  4. CentOS下查看文件和文件夹大小

    当磁盘大小超过标准时会有报警提示,这时如果掌握df和du命令是非常明智的选择. df可以查看一级文件夹大小.使用比例.档案系统及其挂入点,但对文件却无能为力. 当磁盘大小超过标准时会有报警提示,这时如 ...

  5. Centos下删除文件后空间并未释放

    [root@DeviceSP /]# df -h Filesystem Size Used Avail Use% Mounted on /dev/vda3 90G 82G 3.8G 96% / tmp ...

  6. C#替换文件中特定字符串，按照原来的编码格式保存

    private void button1_Click(object sender, EventArgs e) { var txt1 = "E:\\Temp\\local"; str ...

  7. IO复制多级目录 控制台输入文件目录然后把目录下java文件复制到 D: 并统计java个数

    package cn.itcast_05; import java.io.BufferedInputStream; import java.io.BufferedOutputStream; impor ...

  8. 【转】【Linux】Linux下统计当前文件夹下的文件个数、目录个数

    [转][Linux]Linux下统计当前文件夹下的文件个数.目录个数 统计当前文件夹下文件的个数,包括子文件夹里的 ls -lR|grep "^-"|wc -l 统计文件夹下目录的 ...

  9. DataTable、List使用groupby进行分组和分组统计；List、DataTable查询筛选方法

    DataTable分组统计: .用两层循环计算,前提条件是数据已经按分组的列排好序的. DataTable dt = new DataTable(); dt.Columns.AddRange(new ...

随机推荐

  1. 记开发个人图书收藏清单小程序开发（九）Web开发——新增图书信息

    书房信息初始化已完成,现在开始处理图书信息新增功能. 主要是实现之前New Razor Pages的后台部分. 新增需要保存的Model:Book.InitSpec.cs /Models/Book.I ...

  2. deepin jdk 安装

    一.安装JDK环境 1 .卸载系统自带的Openjdk(可能不需要) 先用检测是否安装了openjdk dpkg -l | grep openjdk 如果安装了, 使用下面命令卸载. sudo apt ...

  3. Java50道经典习题-程序11 求不重复数字

    题目:有1.2.3.4个数字,能组成多少个互不相同且无重复数字的三位数?都是多少?分析:可填在百位.十位.个位的数字都是1.2.3.4.组成所有的排列后再去 掉不满足条件的排列. public cla ...

  4. WORD wFormatTag; /* format type */ //设置波形声音的格式

    0x0000   Microsoft   Unknown   Wave   Format       0x0001   Microsoft   PCM   Format       0x0002    ...

  5. java使用Redis6--sentinel单点故障主从自动切换

    Redis SentinelSentinel(哨兵)是用于监控redis集群中Master状态的工具,其已经被集成在redis2.4+的版本中 一.Sentinel作用:1):Master状态检测 2 ...

  6. SAE实践——创建简单留言板

    1. 创建数据库 将mysaeapptest的数据库初始化之后,建立新的数据库,在PHPMyAdmin输入下列SQL. CREATE TABLE IF NOT EXISTS `ny_diary` ( ...

  7. C#-WebForm JS定时器

    JS定时器: 1.window.setTimeout(function(){},3000) 延迟3秒执行 2.window.setInterval(function(){},3000) 也叫重复器,每 ...

  8. TX2 之tensorflow环境部署

    刷机jetpack3.3 首先TX2必须是3.3版本的jetpack,因为截止目前nvidia发布的tensorflow只支持3.3版本的jetpack,刷机的具体步骤可以参考NVIDIA Jetso ...

  9. php 前台生成多维数组 后台批量添加

    同一个地方绊倒两次,记录一下哈 1)前台表单,看 name <div class="tab-pane row " id="tab-1" > < ...

  10. 常见的错误：FTP连接时出现“227 Entering Passive Mode”

    FTP的主动模式(PORT Mode)及被动模式(Passive Mode) FTP的特殊性: 大多数的TCP服务是使用单个的连接,一般是客户向服务器的一个周知端口发起连接,然后使用这个连接进行通讯. ...