布朗语料库中条件概率分布函数ConditionalFreqDist使用

布朗语料库中使用条件概率分布函数ConditionalFreqDist，可以查看每个单词在各新闻语料中出现的次数。这在微博情感分析中非常有用，比如判断feature vector中代表positive or negative or neutral的各feature在每条tweet中出现的次数高低来判断该tweet的情感极性。

from nltk.corpus import brown

cfd=nltk.ConditionalFreqDist(
(genre,word)
for genre in brown.categories()
for word in brown.words(categories=genre)
)
genres=['news','religion','hobbies','science_fiction','romance','humor']
modals=['can','could','may','might','must','will']
print cfd.tabulate(conditions=genres,samples=modals)

输出结果：

can could may might must will
news 93 86 66 38 50 389
religion 82 59 78 12 54 71
hobbies 268 58 131 22 83 264
science_fiction 16 49 4 12 8 16
romance 74 193 11 51 45 43
humor 16 30 8 8 9 13
可以看出news分类中will一词出现最多，humor分类中could出现次数最多。