python简单词频统计

任务

简单统计一个小说中哪些个汉字出现的频率最高

知识点

1. 文件操作
2. 字典
3. 排序
4. lambda

代码

import codecs
import matplotlib.pyplot as plt
from pylab import mpl
mpl.rcParams['font.sans-serif'] = ['FangSong']  # 指定默认字体
mpl.rcParams['axes.unicode_minus'] = False  # 解决保存图像是负号'-'显示为方块的问题

word = []
counter = {}

with codecs.open('data.txt') as fr:
    for line in fr:
        line = line.strip()
        if len(line) == 0:
            continue
        for w in line:
            if not w in word:
                word.append(w)
            if not w in counter:
                counter[w] = 0
            else:
                counter[w] += 1

counter_list = sorted(counter.items(), key=lambda x: x[1], reverse=True)

print(counter_list[:50])

label = list(map(lambda x: x[0], counter_list[:50]))
value = list(map(lambda y: y[1], counter_list[:50]))

plt.bar(range(len(value)), value, tick_label=label)
plt.show()

统计了一个11M的小说，结果如下：

[('，', 288508), ('。', 261584), ('的', 188693), ('陈', 92565), ('欢', 92505), ('不', 91234), ('是', 90562), ('了', 86931), ('一', 79059), ('着', 77997), ('他'
, 71695), ('这', 63580), ('人', 61210), ('“', 59719), ('”', 59115), ('有', 56054), ('就', 52862), ('个', 49097), ('都', 46850), ('你', 45400), ('来', 42659),
 ('我', 40057), ('在', 37676), ('们', 36966), ('到', 36351), ('说', 35828), ('还', 35260), ('么', 32601), ('下', 31742), ('地', 30692), ('得', 29904), ('上', 2
9627), ('看', 28408), ('没', 28333), ('出', 27937), ('道', 27732), ('大', 27012), ('？', 26729), ('那', 26589), ('要', 26076), ('子', 25035), ('自', 24012), ('
点', 23942), ('好', 21345), ('想', 21242), ('里', 20915), ('面', 20661), ('她', 20313), ('过', 20304), ('话', 20110)]