python 自然语言处理（五）____WordNet

WordNet是面向语义的英语词典，与传统辞典类似，但结构更丰富。nltk中包括英语WordNet，共有155287个单词和117659个同义词。

1.寻找同义词

这里以motorcar为例，寻找它的同义词集。

 >>> from nltk.corpus import wordnet as wn
 >>> wn.synsets('motorcar')                                //找到同义词集
 [Synset('car.n.01')]
 >>> wn.synset('car.n.01').lemma_names
 <bound method Synset.lemma_names of Synset('car.n.01')>
 >>> wn.synset('car.n.01').lemma_names()                   //访问同义词集
 ['car', 'auto', 'automobile', 'machine', 'motorcar']
 >>>

 >>> wn.synset('car.n.01').definition()              //获取该词在该词集的定义
 'a motor vehicle with four wheels; usually propelled by an internal combustion engine'
 >>> wn.synset('car.n.01').examples()            //获取该词在该词集下的例句
 ['he needs a car to get to work']
 >>> wn.synset('car.n.01').lemmas()
 [Lemma('car.n.01.car'), Lemma('car.n.01.auto'), Lemma('car.n.01.automobile'), Lemma('car.n.01.machine'), Lemma('car.n.01.motorcar')]
 >>> wn.lemma('car.n.01.automobile')
 Lemma('car.n.01.automobile')
 >>> wn.lemma('car.n.01.automobile').synset()
 Synset('car.n.01')
 >>> wn.lemma('car.n.01.automobile').name()
 'automobile'
 >>> wn.synsets('car')
 [Synset('car.n.01'), Synset('car.n.02'), Synset('car.n.03'), Synset('car.n.04'), Synset('cable_car.n.01')]
 >>> for synset in wn.synsets('car'):
 ...     print (synset.lemma_names())
 ...
 ['car', 'auto', 'automobile', 'machine', 'motorcar']
 ['car', 'railcar', 'railway_car', 'railroad_car']
 ['car', 'gondola']
 ['car', 'elevator_car']
 ['cable_car', 'car']
 >>> wn.lemmas('car')                          //访问所有包含词car的词条
 [Lemma('car.n.01.car'), Lemma('car.n.02.car'), Lemma('car.n.03.car'), Lemma('car.n.04.car'), Lemma('cable_car.n.01.car')]
 >>>

2.WordNet的层次结构

WordNet的同义词集相当于抽象的概念，它们并不总是有对应的英语词汇。这些概念在层次结构中相互联系在一起。

如上图，是WordNet概念的层次片段。每个节点对应一个同义词集，边表示上位词/下位词关系，即上级概念与从属概念的关系。

 >>> motorcar=wn.synset('car.n.01')
 >>> types_of_motorcar=motorcar.hyponyms()
 >>> types_of_motorcar[26]
 Synset('stanley_steamer.n.01')
 >>> sorted(
 ... [lemma.name()
 ... for synset in types_of_motorcar
 ... for lemma in synset.lemmas()])
 ['Model_T', 'S.U.V.', 'SUV', 'Stanley_Steamer', 'ambulance', 'beach_waggon', 'beach_wagon', 'bus', 'cab', 'compact', 'compact_car', 'convert
 ible', 'coupe', 'cruiser', 'electric', 'electric_automobile', 'electric_car', 'estate_car', 'gas_guzzler', 'hack', 'hardtop', 'hatchback', '
 heap', 'horseless_carriage', 'hot-rod', 'hot_rod', 'jalopy', 'jeep', 'landrover', 'limo', 'limousine', 'loaner', 'minicar', 'minivan', 'pace
 _car', 'patrol_car', 'phaeton', 'police_car', 'police_cruiser', 'prowl_car', 'race_car', 'racer', 'racing_car', 'roadster', 'runabout', 'sal
 oon', 'secondhand_car', 'sedan', 'sport_car', 'sport_utility', 'sport_utility_vehicle', 'sports_car', 'squad_car', 'station_waggon', 'statio
 n_wagon', 'stock_car', 'subcompact', 'subcompact_car', 'taxi', 'taxicab', 'tourer', 'touring_car', 'two-seater', 'used-car', 'waggon', 'wago
 n']
 >>> motorcar.hypernyms()
 [Synset('motor_vehicle.n.01')]
 >>> paths=motorcar.hypernym_paths()
 >>> len(paths)
 2
 >>> [synset.name for synset in paths[0]]
 [<bound method Synset.name of Synset('entity.n.01')>, <bound method Synset.name of Synset('physical_entity.n.01')>, <bound method Synset.nam
 e of Synset('object.n.01')>, <bound method Synset.name of Synset('whole.n.02')>, <bound method Synset.name of Synset('artifact.n.01')>, <bou
 nd method Synset.name of Synset('instrumentality.n.03')>, <bound method Synset.name of Synset('container.n.01')>, <bound method Synset.name
 of Synset('wheeled_vehicle.n.01')>, <bound method Synset.name of Synset('self-propelled_vehicle.n.01')>, <bound method Synset.name of Synset
 ('motor_vehicle.n.01')>, <bound method Synset.name of Synset('car.n.01')>]
 >>> [synset.name() for synset in paths[0]]
 ['entity.n.01', 'physical_entity.n.01', 'object.n.01', 'whole.n.02', 'artifact.n.01', 'instrumentality.n.03', 'container.n.01', 'wheeled_veh
 icle.n.01', 'self-propelled_vehicle.n.01', 'motor_vehicle.n.01', 'car.n.01']
 >>> [synset.name() for synset in paths[1]]
 ['entity.n.01', 'physical_entity.n.01', 'object.n.01', 'whole.n.02', 'artifact.n.01', 'instrumentality.n.03', 'conveyance.n.03', 'vehicle.n.
 01', 'wheeled_vehicle.n.01', 'self-propelled_vehicle.n.01', 'motor_vehicle.n.01', 'car.n.01']
 >>> motorcar.root_hypernyms()
 [Synset('entity.n.01')]
 >>>

3.更多的词汇关系

上位词和下位词被称为词汇关系，因为它们是同义集之间的关系。这两者的关系为上下定位“is-a”层次。WordNet网络另一个重要的定位方式是从条目到它们的部件（部分）或到包含它们的东西（整体）。

1）部分-整体关系

 >>> wn.synset('tree.n.01').part_meronyms()
 [Synset('burl.n.02'), Synset('crown.n.07'), Synset('limb.n.02'), Synset('stump.n.01'), Synset('trunk.n.01')]
 >>> wn.synset('tree.n.01').substance_meronyms()
 [Synset('heartwood.n.01'), Synset('sapwood.n.01')]
 >>> wn.synset('tree.n.01').member_holonyms()
 [Synset('forest.n.01')]
 >>> for synset in wn.synsets('mint', wn.NOUN):
 ...     print("%s : %s" % (synset.name(), synset.definition())
 ...
 ...
 ... )
 ...
 batch.n.02 : (often followed by `of') a large number or amount or extent
 mint.n.02 : any north temperate plant of the genus Mentha with aromatic leaves and small mauve flowers
 mint.n.03 : any member of the mint family of plants
 mint.n.04 : the leaves of a mint plant used fresh or candied
 mint.n.05 : a candy that is flavored with a mint oil
 mint.n.06 : a plant where money is coined by authority of the government
 >>> wn.synset('mint.n.04').part_holonyms()
 [Synset('mint.n.02')]
 >>> wn.synset('mint.n.04').substance_holonyms()
 [Synset('mint.n.05')]

2）蕴涵关系

 >>> wn.synset('walk.v.01').entailments()
 [Synset('step.v.01')]
 >>> wn.synset('eat.v.01').entailments()
 [Synset('chew.v.01'), Synset('swallow.v.01')]
 >>> wn.synset('tease.v.03').entailments()
 [Synset('arouse.v.07'), Synset('disappoint.v.01')]

3）反义词

 >>> wn.lemma('supply.n.02.supply').antonyms()
 [Lemma('demand.n.02.demand')]
 >>> wn.lemma('rush.v.01.rush').antonyms()
 [Lemma('linger.v.04.linger')]
 >>> wn.lemma('horizontal.a.01.horizontal').antonyms()
 [Lemma('inclined.a.02.inclined'), Lemma('vertical.a.01.vertical')]
 >>> wn.lemma('staccato.r.01.staccato').antonyms()
 [Lemma('legato.r.01.legato')]
 >>>

4. 语义相似度

同义词集是由复杂的词汇关系网络所连接起来的。给定一个同义词集，可以遍历WordNet网络来查找相关含义的同义词集。每个同义词集都有一个或多个上位词路径连接到一个根上位词。连接到同一个根的两个同义词集可能有一些共同的上位词。如果两个同义词集共用一个特定的上位词——在上位词层次结构中处于较底层——它们一定有密切的联系。