NLTK的安装

一、NLTK：Natural Language Toolkit（自然语言工具包）

　下载：http://www.nltk.org

pip install nltk

二、使用

import nltk
nltk.download()#下载数据

import nltk

text = 'Hello, Tom! How are you recently?'

sens = nltk.sent_tokenize(text) #对文本按照句子进行分割
sens#['Hello, Tom!', 'How are you recently?']

words = []
for sen in sens:
    words.append(nltk.word_tokenize(sen))#对句子进行分词

words#[['Hello', ',', 'Tom', '!'], ['How', 'are', 'you', 'recently', '?']]

tags = []

for tokens in words:
    tags.append(nltk.pos_tag(tokens))#对句子进行词性标注
tags#[[('Hello', 'NNP'), (',', ','), ('Tom', 'NNP'), ('!', '.')], [('How', 'WRB'), ('are', 'VBP'), ('you', 'PRP'), ('recently', 'RB'), ('?', '.')]]

三、安装成功，导入报错

　已经成功安装nltk，但是import nltk时报错：No module named '_sqlite3'

　背景：linux系统自带的python2，已经成功安装nltk，本人自己安装了python3，import nltk出错

　解决方法：sudo apt-get install  sqlite*之后，重新安装python3

#step1
sudo apt-get install  sqlite*

#step2
./configure --prefix=/python3_path
make && make install