自然语言17_Chinking with NLTK
sklearn实战-乳腺癌细胞数据挖掘(博主亲自录制视频教程)
https://study.163.com/course/introduction.htm?courseId=1005269003&utm_campaign=commission&utm_source=cp-400000000398149&utm_medium=share
https://www.pythonprogramming.net/chinking-nltk-tutorial/?completed=/chunking-nltk-tutorial/
代码
# -*- coding: utf-8 -*-
"""
Created on Sun Nov 13 09:14:13 2016 @author: daxiong
"""
import nltk
from nltk.corpus import state_union
from nltk.tokenize import PunktSentenceTokenizer #训练数据
train_text=state_union.raw("2005-GWBush.txt")
#测试数据
sample_text=state_union.raw("2006-GWBush.txt")
'''
Punkt is designed to learn parameters (a list of abbreviations, etc.)
unsupervised from a corpus similar to the target domain.
The pre-packaged models may therefore be unsuitable:
use PunktSentenceTokenizer(text) to learn parameters from the given text
'''
#我们现在训练punkttokenizer(分句器)
custom_sent_tokenizer=PunktSentenceTokenizer(train_text)
#训练后,我们可以使用punkttokenizer(分句器)
tokenized=custom_sent_tokenizer.tokenize(sample_text) '''
nltk.pos_tag(["fire"]) #pos_tag(列表)
Out[19]: [('fire', 'NN')]
'''
'''
#测试语句
words=nltk.word_tokenize(tokenized[0])
tagged=nltk.pos_tag(words)
chunkGram=r"""Chunk:{<RB.?>*<VB.?>*<NNP>+<NN>?}"""
chunkParser=nltk.RegexpParser(chunkGram)
chunked=chunkParser.parse(tagged)
#lambda t:t.label()=='Chunk' 包含Chunk标签的列
for subtree in chunked.subtrees(filter=lambda t:t.label()=='Chunk'):
print(subtree)
''' #文本词性标记函数
def process_content():
try:
for i in tokenized[0:5]:
words = nltk.word_tokenize(i)
tagged = nltk.pos_tag(words) chunkGram = r"""Chunk: {<.*>+}
}<VB.?|IN|DT|TO>+{""" chunkParser = nltk.RegexpParser(chunkGram)
chunked = chunkParser.parse(tagged) chunked.draw() except Exception as e:
print(str(e)) process_content()
百度文库参考
http://wenku.baidu.com/link?url=YIrqeVS8a1zO_H0t66kj1AbUUReLUJIqId5So5Szk0JJAupyg_m2U_WqxEHqAHDy9DfmoAAPu0CdNFf-rePBsTHkx-0WDpoYTH1txFDKQxC
chinking可用于提取句子主干,去除不需要的修饰语
Chinking with NLTK
You may find that, after a lot of chunking, you have some words in
your chunk you still do not want, but you have no idea how to get rid
of them by chunking. You may find that chinking is your solution.
Chinking is a lot like chunking, it is basically a way for you to
remove a chunk from a chunk. The chunk that you remove from your chunk
is your chink.
The code is very similar, you just denote the chink, after the chunk, with }{ instead of the chunk's {}.
import nltk
from nltk.corpus import state_union
from nltk.tokenize import PunktSentenceTokenizer train_text = state_union.raw("2005-GWBush.txt")
sample_text = state_union.raw("2006-GWBush.txt") custom_sent_tokenizer = PunktSentenceTokenizer(train_text) tokenized = custom_sent_tokenizer.tokenize(sample_text) def process_content():
try:
for i in tokenized[5:]:
words = nltk.word_tokenize(i)
tagged = nltk.pos_tag(words) chunkGram = r"""Chunk: {<.*>+}
}<VB.?|IN|DT|TO>+{""" chunkParser = nltk.RegexpParser(chunkGram)
chunked = chunkParser.parse(tagged) chunked.draw() except Exception as e:
print(str(e)) process_content()
With this, you are given something like:
Now, the main difference here is:
}<VB.?|IN|DT|TO>+{
此句表示,我们移除一个或多个动词,介词,定冠词,或to
This means we're removing from the chink one or more verbs, prepositions, determiners, or the word 'to'.
Now that we've learned how to do some custom forms of chunking, and chinking, let's discuss a built-in form of chunking that comes with NLTK, and that is named entity recognition.
自然语言17_Chinking with NLTK的更多相关文章
- 转 --自然语言工具包(NLTK)小结
原作者:http://www.cnblogs.com/I-Tegulia/category/706685.html 1.自然语言工具包(NLTK) NLTK 创建于2001 年,最初是宾州大学计算机与 ...
- 自然语言22_Wordnet with NLTK
QQ:231469242 欢迎喜欢nltk朋友交流 https://www.pythonprogramming.net/wordnet-nltk-tutorial/?completed=/nltk-c ...
- 自然语言16_Chunking with NLTK
Chunking with NLTK 对chunk分类数据结构可以图形化输出,用于分析英语句子主干结构 # -*- coding: utf-8 -*-"""Created ...
- Python自然语言处理工具NLTK的安装FAQ
1 下载Python 首先去python的主页下载一个python版本http://www.python.org/,一路next下去,安装完毕即可 2 下载nltk包 下载地址:http://www. ...
- Python自然语言工具包(NLTK)入门
在本期文章中,小生向您介绍了自然语言工具包(Natural Language Toolkit),它是一个将学术语言技术应用于文本数据集的 Python 库.称为“文本处理”的程序设计是其基本功能:更深 ...
- Python NLTK 自然语言处理入门与例程(转)
转 https://blog.csdn.net/hzp666/article/details/79373720 Python NLTK 自然语言处理入门与例程 在这篇文章中,我们将基于 Pyt ...
- NLTK在自然语言处理
nltk-data.zip 本文主要是总结最近学习的论文.书籍相关知识,主要是Natural Language Pracessing(自然语言处理,简称NLP)和Python挖掘维基百科Infobox ...
- Python自然语言处理工具小结
Python自然语言处理工具小结 作者:白宁超 2016年11月21日21:45:26 目录 [Python NLP]干货!详述Python NLTK下如何使用stanford NLP工具包(1) [ ...
- 自然语言处理(NLP)入门学习资源清单
Melanie Tosik目前就职于旅游搜索公司WayBlazer,她的工作内容是通过自然语言请求来生产个性化旅游推荐路线.回顾她的学习历程,她为期望入门自然语言处理的初学者列出了一份学习资源清单. ...
随机推荐
- Android复习笔记--架构与版本
#Android架构: 1. Linux 内核层 Android 系统是基于Linux 2.6 内核的,这一层为Android 设备的各种硬件提供了底 层的驱动,如显示驱动.音频驱动.照相机驱动.蓝牙 ...
- 【BZOJ 1875】【SDOI 2009】HH去散步
水啊水,最后ans别忘了%哦! #include<cstdio> #include<cstring> #include<algorithm> using names ...
- Maven-通过命令操作maven项目
用Maven 命令创建一个简单的Maven项目 在cmd中运行如下命令: mvn archetype:generate -DgroupId=com.mycompany.app -DartifactId ...
- Eclipse中使用SVN
1.在Eclipse里下载Subclipse插件 方法一:从Eclipse Marketplace里面下载 具体操作:打开Eclipse --> Help --> Eclipse Ma ...
- Segment-Based stereo matching
首先代码实现是根据"Segment-Based Stereo Matching Using Belief Propogation and Self-Adapting Dissimilarit ...
- Leetcode 137. Single Number I/II/III
Given an array of integers, every element appears twice except for one. Find that single one. 本题利用XO ...
- Jenkins使用jenkins-cli.jar进行远程调用时出现“ERROR: No such job 'test'”的问题解决(Windows)
网上最提倡的解决办法是用SSH的key进行登录,但是我发觉Linux上非常容易实现,但是Windows压根不知道在哪里设置. 原文:https://issues.jenkins-ci.org/brow ...
- iOS开发者账号配置进行设备调试
PS:我特么写了这么久,居然图片消失了,服了. 问题一:苹果开发者账号类型: 分为三种:个人的(99美金一年).组织的(99美金一年)和企业账号(299美金一年),申请时需要信用卡,可以找淘宝的代理申 ...
- TCP/IP详解 笔记十三
TCP协议(一) 概述 特点 1, 面向连接可靠的字节流服务 2, 只有两方通信,不能用于广播或多播 3, 应用数据被TCP分隔为最合适发送的数据段,传给IP协议栈 4, 发送端并启动定时器, ...
- JSP+JavaBean+Servlet实现各类列表分页功能
需求: 每页列表下都有一个分页的功能,显示总数量.当前页/总页数.首页.上一页.下一页.最后一页.GO到第几页 效果动态图: 实现思路: 因为每个列表页都需要,在每个出列表页数据的servlet中都要 ...