Linux提取日志中的IP地址并去重
原日志文件：logger.log 内容：
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:30","content":{"ip":"192.168.6.177","method":"GET","uri":"\/kts\/nts\/info","body":"","httpCode":200,"respond":"{\"code\":0,\"msg\":\"成功\",\"result\":{\"type\":1,\"utc\":1686724950400374,\"ltc\":1686724950404128,\"gpsStatus\":1,\"cdmaStatus\":0,\"ktMode\":6,\"ktStatus\":1,\"ftMode\":1,\"gpsEnable\":1,\"cdmaEnable\":0}}"}}
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:30","content":{"ip":"192.168.6.177","method":"GET","uri":"\/kts\/nts\/info","body":"","httpCode":200,"respond":"{\"code\":0,\"msg\":\"成功\",\"result\":{\"type\":1,\"utc\":1686724950511202,\"ltc\":1686724950514957,\"gpsStatus\":1,\"cdmaStatus\":0,\"ktMode\":6,\"ktStatus\":1,\"ftMode\":1,\"gpsEnable\":1,\"cdmaEnable\":0}}"}}
{"peer":"kts_001","type":"ntp","date":"2023-06-14 14:42:30","content":{"clientIp":"192.168.6.177","clientPort":45189,"referenceId":5460039,"rootDispersion":76,"referenceTimestamp":1686724857229845,"originatorTimestamp":1686724952319001,"receiveTimestamp":1686724950601704,"transmitTimestamp":1686724950601921}}
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:30","content":{"ip":"192.168.6.177","method":"GET","uri":"\/kts\/nts\/info","body":"","httpCode":200,"respond":"{\"code\":0,\"msg\":\"成功\",\"result\":{\"type\":1,\"utc\":1686724950604910,\"ltc\":1686724950608664,\"gpsStatus\":1,\"cdmaStatus\":0,\"ktMode\":6,\"ktStatus\":1,\"ftMode\":1,\"gpsEnable\":1,\"cdmaEnable\":0}}"}}
{"peer":"kts_001","type":"ntp","date":"2023-06-14 14:42:30","content":{"clientIp":"192.168.6.177","clientPort":37480,"referenceId":5460039,"rootDispersion":76,"referenceTimestamp":1686724857229845,"originatorTimestamp":1686724952416000,"receiveTimestamp":1686724950698109,"transmitTimestamp":1686724950698350}}
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:30","content":{"ip":"192.168.6.177","method":"GET","uri":"\/kts\/nts\/info","body":"","httpCode":200,"respond":"{\"code\":0,\"msg\":\"成功\",\"result\":{\"type\":1,\"utc\":1686724950700964,\"ltc\":1686724950704719,\"gpsStatus\":1,\"cdmaStatus\":0,\"ktMode\":6,\"ktStatus\":1,\"ftMode\":1,\"gpsEnable\":1,\"cdmaEnable\":0}}"}}
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:30","content":{"ip":"192.168.6.177","method":"GET","uri":"\/kts\/satellite\/info","body":"","httpCode":200,"respond":"{\"code\":0,\"msg\":\"成功\",\"result\":{\"gpsInviewCount\":13,\"bdInviewCount\":11,\"gpsInuseCount\":12,\"bdInuseCount\":13,\"latitude\":4001.4611799999998,\"latitudeHemisphere\":78,\"longitude\":11625.88177,\"longitudeHemisphere\":69,\"gpsSatelliteList\":[{\"id\":4,\"angleElevation\":6,\"angleAzimuth\":33,\"nsRatio\":35},{\"id\":5,\"angleElevation\":42,\"angleAzimuth\":245,\"nsRatio\":42},{\"id\":6,\"angleElevation\":44,\"angleAzimuth\":81,\"nsRatio\":42},{\"id\":9,\"angleElevation\":29,\"angleAzimuth\":53,\"nsRatio\":39},{\"id\":11,\"angleElevation\":70,\"angleAzimuth\":353,\"nsRatio\":44},{\"id\":12,\"angleElevation\":25,\"angleAzimuth\":248,\"nsRatio\":38},{\"id\":17,\"angleElevation\":7,\"angleAzimuth\":147,\"nsRatio\":21},{\"id\":19,\"angleElevation\":25,\"angleAzimuth\":148,\"nsRatio\":39},{\"id\":20,\"angleElevation\":68,\"angleAzimuth\":270,\"nsRatio\":42},{\"id\":25,\"angleElevation\":19,\"angleAzimuth\":286,\"nsRatio\":36},{\"id\":29,\"angleElevation\":14,\"angleAzimuth\":318,\"nsRatio\":27},{\"id\":50,\"angleElevation\":42,\"angleAzimuth\":164,\"nsRatio\":37}],\"bdSatelliteList\":[{\"id\":1,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":31},{\"id\":3,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":27},{\"id\":4,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":28},{\"id\":5,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":28},{\"id\":6,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":16},{\"id\":7,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":15},{\"id\":8,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":14},{\"id\":10,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":16},{\"id\":11,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":15},{\"id\":12,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":16},{\"id\":13,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":13},{\"id\":14,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":15},{\"id\":15,\"angleElevation\":0,\"angleAzimuth\":0,\"nsRatio\":22}]}}"}}
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:30","content":{"ip":"192.168.6.177","method":"GET","uri":"\/kts\/system\/role\/get","body":"","httpCode":200,"respond":"{\"code\":0,\"msg\":\"成功\",\"result\":{\"timeSrc\":\"GPS\",\"level\":1,\"isOK\":true}}"}}
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:30","content":{"ip":"192.168.6.177","method":"GET","uri":"\/kts\/nts\/info","body":"","httpCode":200,"respond":"{\"code\":0,\"msg\":\"成功\",\"result\":{\"type\":1,\"utc\":1686724950708567,\"ltc\":1686724950712321,\"gpsStatus\":1,\"cdmaStatus\":0,\"ktMode\":6,\"ktStatus\":1,\"ftMode\":1,\"gpsEnable\":1,\"cdmaEnable\":0}}"}}
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:33","content":{"ip":"127.0.0.1","method":"GET","uri":"\/kts\/nts\/version","body":"","httpCode":200,"respond":"3.1.7"}}
{"peer":"kts_001","type":"http","date":"2023-06-14 14:42:38","content":{"ip":"127.0.0.1","method":"GET","uri":"\/kts\/nts\/version","body":"","httpCode":200,"respond":"3.1.7"}}
.....提取所有type为ntp的日志
cat /tmp/logger.log |grep ntp > ntp.log得到一个ntp.log文件
{"peer":"kts_001","type":"ntp","date":"2023-06-14 14:42:30","content":{"clientIp":"192.168.6.177","clientPort":45189,"referenceId":5460039,"rootDispersion":76,"referenceTimestamp":1686724857229845,"originatorTimestamp":1686724952319001,"receiveTimestamp":1686724950601704,"transmitTimestamp":1686724950601921}}
{"peer":"kts_001","type":"ntp","date":"2023-06-14 14:42:30","content":{"clientIp":"192.168.6.177","clientPort":37480,"referenceId":5460039,"rootDispersion":76,"referenceTimestamp":1686724857229845,"originatorTimestamp":1686724952416000,"receiveTimestamp":1686724950698109,"transmitTimestamp":1686724950698350}}
......提取IP地址并根据IP去重
cat ntp.log |grep -o '\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}'|sort -d | uniq >> ip.txt得到一个ip.txt文件，里面是所有去重过的ip地址。
Linux提取日志中的IP地址并去重的更多相关文章
- logresolve - 解析Apache日志中的IP地址为主机名
logresolve是一个解析Apache访问日志中IP地址的后处理程序. 为了使对名称服务器的影响降到最低,logresolve拥有极为自主的内部散列表缓存, 使每个IP值仅仅在第一次从日志文件中读 ...
- Java正则表达式提取String字符串中的IP地址
/** * 正则提前字符串中的IP地址 * @param ipString * @return */ public static List<String> getIps(String ip ...
- Python统计日志中每个IP出现次数
介绍了Python统计日志中每个IP出现次数的方法,实例分析了Python基于正则表达式解析日志文件的相关技巧,需要的朋友可以参考下 本脚本可用于多种日志类型 #-*- coding:utf-8 -* ...
- Linux环境下如何配置IP地址、MAC地址
Linux环境下如何配置IP地址.MAC地址 1.配置IP地址 进入配置IP地址路径,进行修改即可 cd /etc/network vim interface 加入以下内容: iface eth0 i ...
- Linux系统下如何设置IP地址？
Linux系统下如何设置IP地址?我们可以通过命令设定IP的方法,不过此方法的前提条件是用户需root权限.在linux系统的 /etc/sysconfig/network-script/ifcfg- ...
- 配置日志中显示IP
package com.demo.conf; import ch.qos.logback.classic.pattern.ClassicConverter; import ch.qos.logback ...
- Linux下获取本机IP地址的代码
Linux下获取本机IP地址的代码,返回值即为互联网标准点分格式的字符串. #define ETH_NAME "eth0" //获得本机IP地址 char* GetLocalAdd ...
- linux(Ubuntu/Centos) iproute 路由IP地址等命令集合，查看端口链接
原 linux(Ubuntu/Centos) iproute 路由IP地址等命令集合,查看端口链接 2017年03月20日 16:55:57 风来了- 阅读数:2291 标签: centoslinux ...
- PHP提取字符串中的图片地址
PHP提取字符串中的图片地址 $str='<p><img border="0" src="upfiles/2009/07/1246430143_1.jp ...
- ubuntu中配置ip地址和上网配置
ubuntu中的ip地址配置: 基于NAT模式: 如果能正常连接ip,但不能ping动www.baidu.com的话,我们需要在配置文件中配置主机上网的DNS地址,如下: 如果上面图片上的DNS se ...
随机推荐
- 安全情报 | Pypi再现窃密攻击投毒
概述 悬镜安全自研的开源组件投毒检测平台通过对主流开源软件仓库(包括Pypi.NPM.Ruby等)发布的组件包进行持续性监控和自动化代码安全分析,同时结合专家安全经验复审,能够及时发现组件包投毒事件并 ...
- mock使用
- 俩小伙一晚上写了个 AI 应用，月入两万？？(文末附开发教程)
开发出一款能够与 AI 对话生成和编辑思维导图的工具,听起来似乎只能是一群专业的 AI 背景团队花费大量的时间和精力训练模型,打磨应用才能完成的事情. 但是,两名大学生却在一夜之间完成了,就像炼金术士 ...
- P2196-DP【黄】
清醒了一点后我又写了一道黄色DP题,做出来了,还行,开心不少了... 中途暴露出一些问题 1.深搜过程中既然用了二维数组,那么深搜时就应该用二维循环取最优解,而不是只从最后一行中进行一维循环取最优解. ...
- C#爬虫知识介绍
爬虫 爬虫(Web Crawler)是指使用程序自动获取互联网上的信息和数据的一种技术手段.它通常从一个起始网址出发,按照一定的规则递归地遍历网页,并将有用的信息提取出来,然后存储到本地或者数据库中, ...
- Java之利用openCsv将csv文件导入mysql数据库
前两天干活儿的时候有个需求,前台导入csv文件,后台要做接收处理,mysql数据库中,项目用的springboot+Vue+mybatisPlus实现,下面详细记录一下实现流程. 1.Controll ...
- java - 递归排序 - 求数组最小值
// 递归求数组最小值 public class Bubble { // 定义存储最小值的变量 static int min; public static void main(String[] arg ...
- [转帖]使用MAT命令行工具生成堆dump分析文件
https://www.cnblogs.com/hellxz/p/use_mat_linux_command_line_generate_reports.html 写作目标 Java程序运行过程中,难 ...
- [转帖]Web技术（六）：QUIC 是如何解决TCP 性能瓶颈的？
文章目录 一.QUIC 如何解决TCP的队头阻塞问题? 1.1 TCP 为何会有队头阻塞问题 1.2 QUIC 如何解决队头阻塞问题 1.3 QUIC 没有队头阻塞的多路复用 二.QUIC 如何优化T ...
- [转帖]KingbaseES不同字符类型比较转换规则
https://www.cnblogs.com/kingbase/p/14798059.html Postgresql 常用的字符数据类型的有char.varchar和text,其中 char 固定长 ...