在分析英文文本时,我们可能会关心文本当中每个词语的词性和在句中起到的作用。识别文本中各个单词词性的过程,可以称为词性标注。

英语主要的八种词性分别为:

1、名词(noun)

2、代词(pronoun)

3、动词(verb)

4、形容词(adjective)

5、副词(adverb)

6、介词(preposition)

7、连词(conjunction)

8、感叹词(interjection)

其他还包括数词(numeral)和冠词(article)等。

在使用第三方工具(如NLTK)进行词性标注时,返回的结果信息量可能比上述八种词性要丰富一些。比如NLTK,其所标注的词性可以参考Penn Treebank Project给出的pos tagset,如下图:

举例来说,我们使用NLTK对一段英文进行词性标注:

这段英文摘自19年3月13日华盛顿邮报有关加拿大停飞波音737客机相关报道,段落的原文为:

After the Lion Air crash, questions were raised, so Boeing sent further instructions that it said pilots should know,” he said, according to the Associated Press. “Those relate to the specific behavior of this specific type of aircraft. As a result, training was given by Boeing, and our pilots have taken it and put it into our manuals.

我们对该段落进行断句,然后对每句话进行分词,再对每个词语进行词性标注,然后循环打印每句话中每个词的词性标注结果,具体代码如下:

 import nltk
passage = """After the Lion Air crash, questions were raised, so Boeing sent further instructions that it said pilots should know,” he said, according to the Associated Press. “Those relate to the specific behavior of this specific type of aircraft. As a result, training was given by Boeing, and our pilots have taken it and put it into our manuals."""
sentences = nltk.sent_tokenize( passage )
for sent in sentences:
tokens = nltk.word_tokenize( sent )
posTags = nltk.pos_tag( tokens )
print( posTags )

代码的print()函数打印的内容如下:

[('After', 'IN'), ('the', 'DT'), ('Lion', 'NNP'), ('Air', 'NNP'), ('crash', 'NN'), (',', ','), ('questions', 'NNS'), ('were', 'VBD'), ('raised', 'VBN'), (',', ','), ('so', 'IN'), ('Boeing', 'NNP'), ('sent', 'VBD'), ('further', 'JJ'), ('instructions', 'NNS'), ('that', 'IN'), ('it', 'PRP'), ('said', 'VBD'), ('pilots', 'NNS'), ('should', 'MD'), ('know', 'VB'), (',', ','), ('”', 'FW'), ('he', 'PRP'), ('said', 'VBD'), (',', ','), ('according', 'VBG'), ('to', 'TO'), ('the', 'DT'), ('Associated', 'NNP'), ('Press', 'NNP'), ('.', '.')]
[('“Those', 'JJ'), ('relate', 'NN'), ('to', 'TO'), ('the', 'DT'), ('specific', 'JJ'), ('behavior', 'NN'), ('of', 'IN'), ('this', 'DT'), ('specific', 'JJ'), ('type', 'NN'), ('of', 'IN'), ('aircraft', 'NN'), ('.', '.')]
[('As', 'IN'), ('a', 'DT'), ('result', 'NN'), (',', ','), ('training', 'NN'), ('was', 'VBD'), ('given', 'VBN'), ('by', 'IN'), ('Boeing', 'NNP'), (',', ','), ('and', 'CC'), ('our', 'PRP$'), ('pilots', 'NNS'), ('have', 'VBP'), ('taken', 'VBN'), ('it', 'PRP'), ('and', 'CC'), ('put', 'VB'), ('it', 'PRP'), ('into', 'IN'), ('our', 'PRP$'), ('manuals', 'NNS'), ('.', '.')]

如何看懂上面的输出结果:段落中的每句话为一个list,每句话中的每个词及其词性表示为一个tuple,左边为单词本身,右边为词性缩写,这些缩写的具体含义可以查找Penn Treebank Pos Tags表格。

我们对代码稍微修改一下,以便使结果呈现更清楚一些,而不至于看的太费力,如下:

 import nltk
passage = """After the Lion Air crash, questions were raised, so Boeing sent further instructions that it said pilots should know,” he said, according to the Associated Press. “Those relate to the specific behavior of this specific type of aircraft. As a result, training was given by Boeing, and our pilots have taken it and put it into our manuals."""
sentences = nltk.sent_tokenize( passage )
for sent in sentences:
tokens = nltk.word_tokenize( sent )
posTags = nltk.pos_tag( tokens )
for tag in posTags:
print( "{}({}) ".format( tag[0], tag[1] ), end = "" )

输出结果如下(标注的词性以括号形式紧跟在每个单词右侧):

After(IN) the(DT) Lion(NNP) Air(NNP) crash(NN) ,(,) questions(NNS) were(VBD) raised(VBN) ,(,) so(IN) Boeing(NNP) sent(VBD) further(JJ) instructions(NNS) that(IN) it(PRP) said(VBD) pilots(NNS) should(MD) know(VB) ,(,) ”(FW) he(PRP) said(VBD) ,(,) according(VBG) to(TO) the(DT) Associated(NNP) Press(NNP) .(.) “Those(JJ) relate(NN) to(TO) the(DT) specific(JJ) behavior(NN) of(IN) this(DT) specific(JJ) type(NN) of(IN) aircraft(NN) .(.) As(IN) a(DT) result(NN) ,(,) training(NN) was(VBD) given(VBN) by(IN) Boeing(NNP) ,(,) and(CC) our(PRP$) pilots(NNS) have(VBP) taken(VBN) it(PRP) and(CC) put(VB) it(PRP) into(IN) our(PRP$) manuals(NNS) .(.) 

参考文献:

1、https://en.wikipedia.org/wiki/Part_of_speech

2、https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html

3、https://www.washingtonpost.com/local/trafficandcommuting/canada-grounds-boeing-737-max-8-leaving-us-as-last-major-user-of-plane/2019/03/13/25ac2414-459d-11e9-90f0-0ccfeec87a61_story.html?utm_term=.f359a714d4d8

POS Tagging 标签类型查询表(Penn Treebank Project)的更多相关文章

  1. html <input>标签类型属性type(file、text、radio、hidden等)详细介绍

    html <input>标签类型属性type(file.text.radio.hidden等)详细介绍 转载请注明:文章转载自:[169IT-最新最全的IT资讯] html <inp ...

  2. 第五篇、HTML标签类型

    <!--1.块级标签 独占一行,可以设置高度和宽度 如:div p h ul li  -----display: none(隐藏标签) block(让行内标签变块级标签) inline(让块级标 ...

  3. POS tagging的解釋

    轉錄文章~~ 什么是词性标注(POS tagging) Tue, 04/13/2010 - 10:36 — Fuller 词性标注也叫词类标注,POS tagging是part-of-speech t ...

  4. 什么是词性标注(POS tagging)

    词性标注也叫词类标注,POS tagging是part-of-speech tagging的缩写. 维基百科对POS Tagging的定义: In corpus linguistics, part-o ...

  5. CSS标签类型和样式表继承与优先级

    标签类型 块级标签 什么是块级标签:在html中<div>. <p>.h1~h6.<form>.<ul> 和 <li>就是块级元素 块级标签 ...

  6. Penn Treebank

    NLP中常用的PTB语料库,全名Penn Treebank.Penn Treebank是一个项目的名称,项目目的是对语料进行标注,标注内容包括词性标注以及句法分析. 语料来源为:1989年华尔街日报语 ...

  7. 创建标签的两种方法insertAdjacentHTML 和 createElement 创建标签 setAttribute 赋予标签类型 appendChild 插入标签

    1. 建立字符串和insertAdjacentHTML('beforeEnd', ) 2. 通过createElement 创建标签  setAttribute 赋予标签类型 appendChild ...

  8. NFC 标签类型

    NFC 标签类型 Type 1:Type 1 Tag is based on ISO/IEC 14443A. This tag type is read and re-write capable. T ...

  9. 伪类的格式重点:父标签层级 & 当前标签类型

    伪类的格式重点:父标签层级 & 当前标签类型 通过例子说明: css1: span:nth-of-type(2){color: red;} css2: span :nth-of-type(2) ...

随机推荐

  1. BootStrap 专题

    验证码的输入框和验证码图片在一行,用bootstrap原生的怎么写呢? 看了教程,没有完全一样的可以让右侧的按钮“输入验证码”固定大小.左侧的输入框动态大小吗?   <div class=&qu ...

  2. nake_api_protect 请求保护器——防止请求被恶意刷

    github : https://github.com/xjnotxj/wechat_interaction_auth -- nake_api_protect 接口请求保护器,根据 频率 + 次数 的 ...

  3. SpringMvc 这篇文章写得不错 多多学习2017.6.29

    http://www.cnblogs.com/bigdataZJ/p/springmvc1.html  博客园链接

  4. java语言为什么能跨平台

    参考https://blog.csdn.net/woailuo453786790/article/details/51660015 因为Java程序编译之后的代码不是能被硬件系统直接运行的代码,而是一 ...

  5. etcd_selector.go

    ) % s.len //not use lock for performance so it is not precise even         server := s.Servers[s.cur ...

  6. Python并发编程之深入理解yield from语法(八)

    大家好,并发编程 进入第八篇. 直到上一篇,我们终于迎来了Python并发编程中,最高级.最重要.当然也是最难的知识点--协程. 当你看到这一篇的时候,请确保你对生成器的知识,有一定的了解.当然不了解 ...

  7. 自动化测试框架 hierarchyViewer、Uiautomator、Appium的区别比较!

    一.HierarchyViewer: 优点:1)能够可视化的角度直观地获得UI布局设计结构和各种属性的信息 2)使用hierarchyviewer.bat来检索手机界面布局 缺点:1)稳定性差,出现过 ...

  8. ASP.NET Core 借助 K8S 玩转容器编排

    Production-Grade Container Orchestration - Automated container deployment, scaling, and management. ...

  9. 【STM32H7教程】第9章 STM32H7重要知识点数据类型,变量和堆栈

    完整教程下载地址:http://forum.armfly.com/forum.php?mod=viewthread&tid=86980 第9章   STM32H7重要知识点数据类型,变量和堆栈 ...

  10. 学习攻略丨如何进阶为一名Web安全高手?

    学习Web安全的小伙伴很多,但是能成为Web安全高手却很少,很多人都是从入门到放弃,是真的太难还是学习方法不对? 对于基础薄弱的人来说,一般都是从XSS.SQL注入等简单的漏洞研究入门的.除了了解各种 ...