linux awk和sed工具

慕课网链接：https://www.imooc.com/video/14508

部分示例命令

#替换passwd中的用户名和userid和gid
gsed 's/\(^[a-z_-]\+\):\*:\([0-9]\+\):\([0-9-]\+\):.*$/user:\1 uid:\2 pid:\3/' passwd
#替换出网卡中的ip地址
ifconfig en0 | gsed -n '/inet /p' | gsed 's/inet \([0-9.]\+\) .*$/\1/'

#Awk
awk -F ':' '{print $1, $3}' passwd
awk -F ':' '{print "User:"$1,"UID:"$3}' passwd
➜  mac-function-test awk -F ':' '{print NR,NF,FILENAME}' passwd

#print和printf
➜  mac-function-test awk -F ':' '{print "Line: "NR, "Col:"Nf,"User:"$1}' passwd
➜  mac-function-test awk -F ':' '{printf("Line:%3s Col:%s User:%s\n",NR,NF,$1)}' passwd
#password文档中用户id大于100的记录
➜  mac-function-test awk -F ':' '{if ($3>100) print "Line: "NR,"User: "$1, "UserId: "$3}' passwd
#打印nginx日志文件中错误发生时间
➜  logs gsed -n '/error/p' error.log | awk '{print $1,$2 }'
打印错误时间方法二
➜  logs awk '/error/{print $1,$2}' error.log
# ~逻辑判断的表达式 !~逻辑判断的表达式 取反 匹配正则表达式
➜  mac-function-test awk -F ':' '$1!~/^e.*/{print $1}' passwd
#userId小于100的数据
➜  mac-function-test awk -F ':' '$3<100{print $1,$3}' passwd

#用passwd文件中的3个值做一个统计表
➜  mac-function-test awk -F ':' 'BEGIN{print "Line  Col  User"}{print NR,NF,$1}END{print"-----"FILENAME"------"}' passwd

#当前目录下文件占用的大小
➜  mac-function-test ls -l | awk 'BEGIN{size=0}{size+=$5}END{print " size is " size/1024/1024"M"}'
#统计passwd的账号总人数
➜  mac-function-test awk -F ':' 'BEGIN{count=0}$1!~/^$/{count++}END{print "count = "count}' passwd
#打印用户id大于100的账号
➜  mac-function-test awk -F ':' 'BEGIN{count=0}{if($3> 100) name[count++]=$1}END{for(i=0;i<count;i++)print i,name[i]}' passwd
#统计netstat -anp状态下为ESTABLISHED

　将.properties文件中注释的行替换为空

gsed -i 's/^#.*//' needupload.properties

　　

bilibili中不错的视频教程：

https://www.bilibili.com/video/BV1WW411v7PS/?p=11

相应推荐的在线网站：

https://regexr.com/

正则注意贪婪模式和转义符号“\”；

慕课网课程2:https://www.imooc.com/video/7345

linux通配符:*代表匹配任意字符；?代表匹配任意一个字符；[]代表匹配其中的一个字符

➜  mac-function-test touch cangls
➜  mac-function-test touch canyls
➜  mac-function-test ls can?ls
cangls canyls
➜  mac-function-test ls can???
cangls canyls
➜  mac-function-test ls can*
cangls canyls
➜  mac-function-test ls can[gy]ls
cangls canyls
➜  mac-function-test ls can[g]ls
cangls
➜  mac-function-test touc abc
zsh: command not found: touc
➜  mac-function-test touch abc
➜  mac-function-test touch abcd
➜  mac-function-test find . -name abc
./abc
➜  mac-function-test find . -name abc?
./abcd
➜  mac-function-test find . -name "abc*"
./abc
./abc.txt
./abcd

　

正则中“*”前一个字符匹配0次，或者任意多次

“a*”

#匹配所有内容，包括空白行

“aa*”

#匹配至少包含一个a的行

正则中“.”匹配除了换行符外的任意一个字符

“[]”匹配中括号中指定的任意一个字符，只匹配一个字符

“^[^a-z]”

#匹配不用小写字母开头的行

“^[^a-zA-Z]”

#匹配不用字母开头的行

“\”转移符

“\{n\}”表示其前面的字符恰好出现n次

#例子 匹配2010-09-01

[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}

awk使用print和printf的区别
➜  mac-function-test awk '{printf $2 "\t" $4 "\n"}' student.txt
➜  mac-function-test awk '{print $2 "\t" $4 }' student.txt
➜  mac-function-test df -h | grep '/' | awk '{print $5}' | cut -d "%" -f 1

➜  mac-function-test cat student.txt
Id	name	gender	mark
1	furong	f	85
2	fengj	F	60
3	cang	F	70
➜  mac-function-test awk '{print $2 "\t" $4}' student.txt
name	mark
furong	85
fengj	60
cang	70
➜  mac-function-test awk 'BEGIN{print "test"}{print $2 "\t" $4}' student.txt
test
name	mark
furong	85
fengj	60
cang	70
➜  mac-function-test awk 'END{print "test"}{print $2 "\t" $4}' student.txt
name	mark
furong	85
fengj	60
cang	70
test

➜  mac-function-test cat passwd|grep /bin/bash
_mbsetupuser:*:248:248:Setup User:/var/setup:/bin/bash
➜  mac-function-test cat passwd|grep /bin/bash |awk '{FS=":"}{print $1 "\t" $3}'
_mbsetupuser:*:248:248:Setup
➜  mac-function-test cat passwd|grep /bin/bash |awk 'BEGIN{FS=":"}{print $1 "\t" $3}'
_mbsetupuser	248
➜  mac-function-test cat student.txt|grep -v name
1	furong	f	85
2	fengj	F	60
3	cang	F	70
➜  mac-function-test cat student.txt|grep -v name | awk '$4>=70{print $2}'
furong
cang

➜  mac-function-test sort -n -t ":" -k 3,3 passwd