初识sed和gwak

一、sed编辑器

sed命令的格式如下：

sed options script file

选项

-e script        在处理输入时，将script中指定的命令添加到已有的命令中

-f file            在处理输入时，将file中指定的命令添加到已有的命令中

-n                不产生命令输出，使用print命令来完成输出



1.在命令行定义编辑器命令

1. $ echo "This is a test" | sed '/test/big test/'
2. This is a big test

1. [root@localhost sed]# vim data1.txt
2. [root@localhost sed]# sed 's/dog/cat/' data1.txt
3. The quick brown fox jumps over the lazy cat.
4. The quick brown fox jumps over the lazy cat.
5. The quick brown fox jumps over the lazy cat.
6. The quick brown fox jumps over the lazy cat.
7. The quick brown fox jumps over the lazy cat.
8. The quick brown fox jumps over the lazy cat.
9. The quick brown fox jumps over the lazy cat.
10. [root@localhost sed]# cat data1.txt
11. The quick brown fox jumps over the lazy dog.
12. The quick brown fox jumps over the lazy dog.
13. The quick brown fox jumps over the lazy dog.
14. The quick brown fox jumps over the lazy dog.
15. The quick brown fox jumps over the lazy dog.
16. The quick brown fox jumps over the lazy dog.
17. The quick brown fox jumps over the lazy dog.

2.在命令行使用多个编辑器命令

1. [root@localhost sed]# sed -e 's/brown/green/; s/dog/cat/' data1.txt
2. The quick green fox jumps over the lazy cat.
3. The quick green fox jumps over the lazy cat.
4. The quick green fox jumps over the lazy cat.
5. The quick green fox jumps over the lazy cat.
6. The quick green fox jumps over the lazy cat.
7. The quick green fox jumps over the lazy cat.
8. The quick green fox jumps over the lazy cat.
9. [root@localhost sed]# cat data1.txt
10. The quick brown fox jumps over the lazy dog.
11. The quick brown fox jumps over the lazy dog.
12. The quick brown fox jumps over the lazy dog.
13. The quick brown fox jumps over the lazy dog.
14. The quick brown fox jumps over the lazy dog.
15. The quick brown fox jumps over the lazy dog.
16. The quick brown fox jumps over the lazy dog.

3.从文件中读取编辑器命令

1. [root@localhost sed]# cat script1.sed
2. s/brown/green/
3. s/fox/elephant/
4. s/dog/cat/
5. [root@localhost sed]# sed -f script1.sed data1.txt
6. The quick green elephant jumps over the lazy cat.
7. The quick green elephant jumps over the lazy cat.
8. The quick green elephant jumps over the lazy cat.
9. The quick green elephant jumps over the lazy cat.
10. The quick green elephant jumps over the lazy cat.
11. The quick green elephant jumps over the lazy cat.
12. The quick green elephant jumps over the lazy cat.

二、gawk程序可以

①定义变量来保存数据；

②使用算数和字符串操作符处理数据；

③使用结构化编程概念(if-then)来为数据处理增减处理逻辑；

④通过提取数据文件中的数据元素，将其重新排列或格式化，生成格式化报告。



gawk是awk语言中使用gnu的一种



2.1 gawk命令格式

gawk options program file

选项

-F fs                                   指定行中划分数据字段分隔符

-f file                                 从指定的文件中读取程序

-v var=value                      定义gawk程序中的一个变量及其默认值

-mf N                                指定要处理的数据文件中的最大字段数

-mr N                                指定数据文件中的最大数据行数

-W keyword                       指定gawk的兼容模式或警告等级



2.2 从命令行读取程序脚本

1. [root@localhost sed]# gawk '{print "Hello World!"}'
2. This is a test
3. Hello World!
4. hello
5. Hello World!
6. This is another test
7. Hello World!
8. Hello World!
9. ^C

2.3 使用数据字段变量

$0代表整个文本行

$1代表文本行中的第1个数据字段

$2代表文本行中的第2个数据字段

$n代表文本行中的第n个数据字段

1. [root@localhost gawk]# cat data2.txt
2. One line of test text.
3. Two line of test text.
4. Three line of test text.
5. [root@localhost gawk]# gawk '{print $1}' data2.txt
6. One
7. Two
8. Three

1. [root@localhost gawk]# gawk -F: '{print $1}' /etc/passwd
2. root
3. bin
4. daemon
5. adm
6. lp
7. sync
8. shutdown
9. halt
10. mail
11. uucp
12. operator
13. games
14. gopher
15. ftp
16. nobody

如果不是空格分隔符，则用-F指定



2.4 在程序中使用多个命令

1. [root@localhost gawk]# echo "My name is Rich" | gawk '{$4="Christine"; print $0}'
2. My name is Christine

2.5 从文件中读取程序

1. [root@localhost gawk]# cat script2.gawk
2. {print $1 "'s home directory is " $6}
3. [root@localhost gawk]# gawk -F: -f script2.gawk /etc/passwd
4. root's home directory is /root
5. bin's home directory is /bin
6. daemon's home directory is /sbin
7. adm's home directory is /var/adm
8. lp's home directory is /var/spool/lpd
9. sync's home directory is /sbin
10. shutdown's home directory is /sbin
11. halt's home directory is /sbin
12. mail's home directory is /var/spool/mail
13. uucp's home directory is /var/spool/uucp
14. operator's home directory is /root
15. games's home directory is /usr/games
16. gopher's home directory is /var/gopher

1. [root@localhost gawk]# cat script3.gawk
2. {
3. text = "'s home directory is "
4. print $1 text $6
5. }
6. [root@localhost gawk]# gawk -F: -f script3.gawk /etc/passwd
7. root's home directory is /root
8. bin's home directory is /bin
9. daemon's home directory is /sbin
10. adm's home directory is /var/adm
11. lp's home directory is /var/spool/lpd
12. sync's home directory is /sbin
13. shutdown's home directory is /sbin

2.6 在处理数据前运行程序脚本

1. [root@localhost gawk]# gawk 'BEGIN {print "Hello World!"}'
2. Hello World!

1. [root@localhost gawk]# cat data3.txt
2. Line 1
3. Line 2
4. Line 3
5. [root@localhost gawk]# gawk 'BEGIN {print "The data3 File Contents:"}
6. {print $0}' data3.txt
7. The data3 File Contents:
8. Line 1
9. Line 2
10. Line 3

2.7 在处理数据后运行脚本

1. [root@localhost gawk]# gawk 'BEGIN {print "The data3 File Contents:"}
2. > {print $0}
3. > END {print "End of file"}' data3.txt
4. The data3 File Contents:
5. Line 1
6. Line 2
7. Line 3
8. End of file

1. [root@localhost gawk]# cat script4.gawk
2. BEGIN {
3. print "The latest list of users and shells"
4. print " UserID \t shell"
5. print "-------- \t -------"
6. FS=":"
7. }
8. {
9. print $1 " \t " $7
10. }
11. END {
12. print "This concludes the listing"
13. }
14. [root@localhost gawk]# gawk -f script4.gawk /etc/passwd
15. The latest list of users and shells
16. UserID shell
17. -------- -------
18. root /bin/bash
19. bin /sbin/nologin
20. daemon /sbin/nologin
21. adm /sbin/nologin
22. lp /sbin/nologin
23. sync /bin/sync
24. 。。。
25. nfsnobody        /sbin/nologin
26. This concludes the listing