标签归档：nltk

知识问答

NLTK可能有哪些pos标签？

问题：NLTK可能有哪些pos标签？

如何找到包含自然语言工具包（nltk）使用的所有可能pos标记的列表？

点击查看英文原文

How do I find a list with all possible pos tags used by the Natural Language Toolkit (nltk)?

回答 0

这本书有一个注释，说明如何在标签集上寻求帮助，例如：

nltk.help.upenn_tagset()

其他人可能相似。（注意：也许您首先需要为此tagsets从下载助手的“ 模型”部分进行下载）

点击查看英文原文

The book has a note how to find help on tag sets, e.g.:

nltk.help.upenn_tagset()

Others are probably similar. (Note: Maybe you first have to download tagsets from the download helper’s Models section for this)

回答 1

为了节省一些时间，这是我从一个小型语料库中提取的列表。我不知道它是否完整，但是它应该具有upenn_tagset中的大多数（如果不是全部）帮助定义…

CC：结合，协调

& 'n and both but either et for less minus neither nor or plus so
therefore times v. versus vs. whether yet

CD：数字，基数

mid-1890 nine-thirty forty-two one-tenth ten million 0.5 one forty-
seven 1987 twenty '79 zero two 78-degrees eighty-four IX '60s .025
fifteen 271,124 dozen quintillion DM2,000 ...

DT：确定者

all an another any both del each either every half la many much nary
neither no some such that the them these this those

EX：存在存在

there

IN：介词或连词，从属

astride among uppon whether out inside pro despite on by throughout
below within for towards near behind atop around if like until below
next into if beside ...

JJ：形容词或数字，序数

third ill-mannered pre-war regrettable oiled calamitous first separable
ectoplasmic battery-powered participatory fourth still-to-be-named
multilingual multi-disciplinary ...

JJR：形容词，比较

bleaker braver breezier briefer brighter brisker broader bumper busier
calmer cheaper choosier cleaner clearer closer colder commoner costlier
cozier creamier crunchier cuter ...

JJS：形容词，最高级

calmest cheapest choicest classiest cleanest clearest closest commonest
corniest costliest crassest creepiest crudest cutest darkest deadliest
dearest deepest densest dinkiest ...

LS：列表项标记

A A. B B. C C. D E F First G H I J K One SP-44001 SP-44002 SP-44005
SP-44007 Second Third Three Two * a b c d first five four one six three
two

MD：模态辅助

can cannot could couldn't dare may might must need ought shall should
shouldn't will would

NN：名词，普通，单数或质量

common-carrier cabbage knuckle-duster Casino afghan shed thermostat
investment slide humour falloff slick wind hyena override subhumanity
machinist ...

NNP：名词，专有名词，单数

Motown Venneboerger Czestochwa Ranzer Conchita Trumplane Christos
Oceanside Escobar Kreisler Sawyer Cougar Yvette Ervin ODI Darryl CTCA
Shannon A.K.C. Meltex Liverpool ...

NNS：名词，常见，复数

undergraduates scotches bric-a-brac products bodyguards facets coasts
divestitures storehouses designs clubs fragrances averages
subjectivists apprehensions muses factory-jobs ...

PDT：预定项

all both half many quite such sure this

POS：遗传标记

' 's

PRP：代词，个人

hers herself him himself hisself it itself me myself one oneself ours
ourselves ownself self she thee theirs them themselves they thou thy us

PRP $：代词，所有格

her his mine my our ours their thy your

RB：副词

occasionally unabatingly maddeningly adventurously professedly
stirringly prominently technologically magisterially predominately
swiftly fiscally pitilessly ...

RBR：副词，比较

further gloomier grander graver greater grimmer harder harsher
healthier heavier higher however larger later leaner lengthier less-
perfectly lesser lonelier longer louder lower more ...

RBS：副词，最高级

best biggest bluntest earliest farthest first furthest hardest
heartiest highest largest least less most nearest second tightest worst

RP：颗粒

aboard about across along apart around aside at away back before behind
by crop down ever fast for forth from go high i.e. in into just later
low more off on open out over per pie raising start teeth that through
under unto up up-pp upon whole with you

TO：“ to”作为介词或不定式标记

to

UH：感叹词

Goodbye Goody Gosh Wow Jeepers Jee-sus Hubba Hey Kee-reist Oops amen
huh howdy uh dammit whammo shucks heck anyways whodunnit honey golly
man baby diddle hush sonuvabitch ...

VB：动词，基本形式

ask assemble assess assign assume atone attention avoid bake balkanize
bank begin behold believe bend benefit bevel beware bless boil bomb
boost brace break bring broil brush build ...

VBD：动词，过去时

dipped pleaded swiped regummed soaked tidied convened halted registered
cushioned exacted snubbed strode aimed adopted belied figgered
speculated wore appreciated contemplated ...

VBG：动词，现在分词或动名词

telegraphing stirring focusing angering judging stalling lactating
hankerin' alleging veering capping approaching traveling besieging
encrypting interrupting erasing wincing ...

VBN：动词，过去分词

multihulled dilapidated aerosolized chaired languished panelized used
experimented flourished imitated reunifed factored condensed sheared
unsettled primed dubbed desired ...

VBP：动词，现在时，不是第三人称单数

predominate wrap resort sue twist spill cure lengthen brush terminate
appear tend stray glisten obtain comprise detest tease attract
emphasize mold postpone sever return wag ...

VBZ：动词，现在时，第三人称单数

bases reconstructs marks mixes displeases seals carps weaves snatches
slumps stretches authorizes smolders pictures emerges stockpiles
seduces fizzes uses bolsters slaps speaks pleads ...

WDT：WH决定因素

that what whatever which whichever

WP：WH代词

that what whatever whatsoever which who whom whosoever

WRB：Wh-副词

how however whence whenever where whereby whereever wherein whereof why
点击查看英文原文

To save some folks some time, here is a list I extracted from a small corpus. I do not know if it is complete, but it should have most (if not all) of the help definitions from upenn_tagset…

CC: conjunction, coordinating

& 'n and both but either et for less minus neither nor or plus so
therefore times v. versus vs. whether yet

CD: numeral, cardinal

mid-1890 nine-thirty forty-two one-tenth ten million 0.5 one forty-
seven 1987 twenty '79 zero two 78-degrees eighty-four IX '60s .025
fifteen 271,124 dozen quintillion DM2,000 ...

DT: determiner

all an another any both del each either every half la many much nary
neither no some such that the them these this those

EX: existential there

there

IN: preposition or conjunction, subordinating

astride among uppon whether out inside pro despite on by throughout
below within for towards near behind atop around if like until below
next into if beside ...

JJ: adjective or numeral, ordinal

third ill-mannered pre-war regrettable oiled calamitous first separable
ectoplasmic battery-powered participatory fourth still-to-be-named
multilingual multi-disciplinary ...

JJR: adjective, comparative

bleaker braver breezier briefer brighter brisker broader bumper busier
calmer cheaper choosier cleaner clearer closer colder commoner costlier
cozier creamier crunchier cuter ...

JJS: adjective, superlative

calmest cheapest choicest classiest cleanest clearest closest commonest
corniest costliest crassest creepiest crudest cutest darkest deadliest
dearest deepest densest dinkiest ...

LS: list item marker

A A. B B. C C. D E F First G H I J K One SP-44001 SP-44002 SP-44005
SP-44007 Second Third Three Two * a b c d first five four one six three
two

MD: modal auxiliary

can cannot could couldn't dare may might must need ought shall should
shouldn't will would

NN: noun, common, singular or mass

common-carrier cabbage knuckle-duster Casino afghan shed thermostat
investment slide humour falloff slick wind hyena override subhumanity
machinist ...

NNP: noun, proper, singular

Motown Venneboerger Czestochwa Ranzer Conchita Trumplane Christos
Oceanside Escobar Kreisler Sawyer Cougar Yvette Ervin ODI Darryl CTCA
Shannon A.K.C. Meltex Liverpool ...

NNS: noun, common, plural

undergraduates scotches bric-a-brac products bodyguards facets coasts
divestitures storehouses designs clubs fragrances averages
subjectivists apprehensions muses factory-jobs ...

PDT: pre-determiner

all both half many quite such sure this

POS: genitive marker

' 's

PRP: pronoun, personal

hers herself him himself hisself it itself me myself one oneself ours
ourselves ownself self she thee theirs them themselves they thou thy us

PRP$: pronoun, possessive

her his mine my our ours their thy your

RB: adverb

occasionally unabatingly maddeningly adventurously professedly
stirringly prominently technologically magisterially predominately
swiftly fiscally pitilessly ...

RBR: adverb, comparative

further gloomier grander graver greater grimmer harder harsher
healthier heavier higher however larger later leaner lengthier less-
perfectly lesser lonelier longer louder lower more ...

RBS: adverb, superlative

best biggest bluntest earliest farthest first furthest hardest
heartiest highest largest least less most nearest second tightest worst

RP: particle

aboard about across along apart around aside at away back before behind
by crop down ever fast for forth from go high i.e. in into just later
low more off on open out over per pie raising start teeth that through
under unto up up-pp upon whole with you

TO: “to” as preposition or infinitive marker

to

UH: interjection

Goodbye Goody Gosh Wow Jeepers Jee-sus Hubba Hey Kee-reist Oops amen
huh howdy uh dammit whammo shucks heck anyways whodunnit honey golly
man baby diddle hush sonuvabitch ...

VB: verb, base form

ask assemble assess assign assume atone attention avoid bake balkanize
bank begin behold believe bend benefit bevel beware bless boil bomb
boost brace break bring broil brush build ...

VBD: verb, past tense

dipped pleaded swiped regummed soaked tidied convened halted registered
cushioned exacted snubbed strode aimed adopted belied figgered
speculated wore appreciated contemplated ...

VBG: verb, present participle or gerund

telegraphing stirring focusing angering judging stalling lactating
hankerin' alleging veering capping approaching traveling besieging
encrypting interrupting erasing wincing ...

VBN: verb, past participle

multihulled dilapidated aerosolized chaired languished panelized used
experimented flourished imitated reunifed factored condensed sheared
unsettled primed dubbed desired ...

VBP: verb, present tense, not 3rd person singular

predominate wrap resort sue twist spill cure lengthen brush terminate
appear tend stray glisten obtain comprise detest tease attract
emphasize mold postpone sever return wag ...

VBZ: verb, present tense, 3rd person singular

bases reconstructs marks mixes displeases seals carps weaves snatches
slumps stretches authorizes smolders pictures emerges stockpiles
seduces fizzes uses bolsters slaps speaks pleads ...

WDT: WH-determiner

that what whatever which whichever

WP: WH-pronoun

that what whatever whatsoever which who whom whosoever

WRB: Wh-adverb

how however whence whenever where whereby whereever wherein whereof why

回答 2

标签集取决于用于训练标签手的语料库。的默认标记器nltk.pos_tag()使用Penn树库标记集。

在NLTK 2中，可以检查哪个标记器是默认标记器，如下所示： 

import nltk
nltk.tag._POS_TAGGER
>>> 'taggers/maxent_treebank_pos_tagger/english.pickle'

这意味着它是在树库语料库上训练的最大熵标记器。

nltk.tag._POS_TAGGERNLTK 3中已不存在这种标记，但是文档指出，现成的标记器仍使用Penn Treebank标签集。

点击查看英文原文

The tag set depends on the corpus that was used to train the tagger. The default tagger of nltk.pos_tag() uses the Penn Treebank Tag Set.

In NLTK 2, you could check which tagger is the default tagger as follows: 

import nltk
nltk.tag._POS_TAGGER
>>> 'taggers/maxent_treebank_pos_tagger/english.pickle'

That means that it’s a Maximum Entropy tagger trained on the Treebank corpus.

nltk.tag._POS_TAGGER does not exist anymore in NLTK 3 but the documentation states that the off-the-shelf tagger still uses the Penn Treebank tagset.

回答 3

以下内容对于访问以缩写键键入的字典非常有用：

>>> from nltk.data import load
>>> tagdict = load('help/tagsets/upenn_tagset.pickle')
>>> tagdict['NN'][0]
'noun, common, singular or mass'
>>> tagdict.keys()
['PRP$', 'VBG', 'VBD', '``', 'VBN', ',', "''", 'VBP', 'WDT', ...
点击查看英文原文

The below can be useful to access a dict keyed by abbreviations:

>>> from nltk.data import load
>>> tagdict = load('help/tagsets/upenn_tagset.pickle')
>>> tagdict['NN'][0]
'noun, common, singular or mass'
>>> tagdict.keys()
['PRP$', 'VBG', 'VBD', '``', 'VBN', ',', "''", 'VBP', 'WDT', ...

回答 4

该参考可在官方网站上找到

从那里复制和粘贴：

  • CC | 协调连词|
  • CD | 基数|
  • DT | 确定者|
  • EX | 生存有 |
  • FW | 外来词|
  • IN | 介词或从属连词|
  • JJ | 形容词|
  • JJR | 形容词，比较
  • JJS | 形容词，最高级
  • LS | 清单项目标记|
  • MD | 模态|
  • NN | 奇异或名词
  • NNS | 名词，复数|
  • NNP | 专有名词，单数|
  • NNPS | 专有名词，复数|
  • PDT | 预定器|
  • POS | 可能的结局|
  • PRP | 人称代词|
  • PRP $ | 所有格代词|
  • RB | 副词|
  • RBR | 比较副词|
  • 苏格兰皇家银行 最高级副词|
  • RP | 颗粒|
  • SYM | 符号
  • 至| 至 |
  • UH | 感叹词|
  • VB | 动词，基础形式|
  • VBD | 动词，过去时|
  • VBG | 动词，动名词或现在分词|
  • VBN | 动词，过去分词|
  • VBP | 动词，非第三人称单数形式的礼物|
  • VBZ | 动词，第三人称单数形式的礼物|
  • 看门狗| Wh决定子|
  • WP | Wh代词|
  • WP $ | 所有制Wh代词|
  • WRB | Wh-副词|
点击查看英文原文

The reference is available at the official site

Copy and pasting from there:

  • CC | Coordinating conjunction |
  • CD | Cardinal number |
  • DT | Determiner |
  • EX | Existential there |
  • FW | Foreign word |
  • IN | Preposition or subordinating conjunction |
  • JJ | Adjective |
  • JJR | Adjective, comparative |
  • JJS | Adjective, superlative |
  • LS | List item marker |
  • MD | Modal |
  • NN | Noun, singular or mass |
  • NNS | Noun, plural |
  • NNP | Proper noun, singular |
  • NNPS | Proper noun, plural |
  • PDT | Predeterminer |
  • POS | Possessive ending |
  • PRP | Personal pronoun |
  • PRP$ | Possessive pronoun |
  • RB | Adverb |
  • RBR | Adverb, comparative |
  • RBS | Adverb, superlative |
  • RP | Particle |
  • SYM | Symbol |
  • TO | to |
  • UH | Interjection |
  • VB | Verb, base form |
  • VBD | Verb, past tense |
  • VBG | Verb, gerund or present participle |
  • VBN | Verb, past participle |
  • VBP | Verb, non-3rd person singular present |
  • VBZ | Verb, 3rd person singular present |
  • WDT | Wh-determiner |
  • WP | Wh-pronoun |
  • WP$ | Possessive wh-pronoun |
  • WRB | Wh-adverb |

回答 5

您可以在此处下载列表：ftp : //ftp.cis.upenn.edu/pub/treebank/doc/tagguide.ps.gz。它包括令人困惑的词性，大写和其他约定。同样，维基百科有一个与此类似的有趣部分。部分：使用的词性标签。

点击查看英文原文

You can download the list here: ftp://ftp.cis.upenn.edu/pub/treebank/doc/tagguide.ps.gz. It includes confusing parts of speech, capitalization, and other conventions. Also, wikipedia has an interesting section similar to this. Section: Part-of-speech tags used.

回答 6

['LS', 'TO', 'VBN', "''", 'WP', 'UH', 'VBG', 'JJ', 'VBZ', '--', 'VBP', 'NN', 'DT', 'PRP', ':', 'WP$', 'NNPS', 'PRP$', 'WDT', '(', ')', '.', ',', '``', '$', 'RB', 'RBR', 'RBS', 'VBD', 'IN', 'FW', 'RP', 'JJR', 'JJS', 'PDT', 'MD', 'VB', 'WRB', 'NNP', 'EX', 'NNS', 'SYM', 'CC', 'CD', 'POS']

基于Doug Shore的方法，但使其更易于复制粘贴

点击查看英文原文 
['LS', 'TO', 'VBN', "''", 'WP', 'UH', 'VBG', 'JJ', 'VBZ', '--', 'VBP', 'NN', 'DT', 'PRP', ':', 'WP$', 'NNPS', 'PRP$', 'WDT', '(', ')', '.', ',', '``', '$', 'RB', 'RBR', 'RBS', 'VBD', 'IN', 'FW', 'RP', 'JJR', 'JJS', 'PDT', 'MD', 'VB', 'WRB', 'NNP', 'EX', 'NNS', 'SYM', 'CC', 'CD', 'POS']

Based on Doug Shore’s method but make it more copy-paste friendly

回答 7

只需逐字运行即可。 

import nltk
nltk.download('tagsets')
nltk.help.upenn_tagset()

nltk.tag._POS_TAGGER将无法正常工作。它将给出AttributeError：模块’nltk.tag’没有属性’_POS_TAGGER’。NLTK 3中不再提供该功能。

点击查看英文原文

Just run this verbatim. 

import nltk
nltk.download('tagsets')
nltk.help.upenn_tagset()

nltk.tag._POS_TAGGER won’t work. It will give AttributeError: module ‘nltk.tag’ has no attribute ‘_POS_TAGGER’. It’s not available in NLTK 3 anymore.

github

TextBlob-简单、Python式的文本处理–情感分析、词性标记、名词短语提取、翻译等等

主页：https://textblob.readthedocs.io/

TextBlob是一个Python(2和3)库，用于处理文本数据。它提供了一个简单的API，用于深入研究常见的自然语言处理(NLP)任务，如词性标记、名词短语提取、情感分析、分类、翻译等

 

from textblob import TextBlob

text = '''
The titular threat of The Blob has always struck me as the ultimate movie
monster: an insatiably hungry, amoeba-like mass able to penetrate
virtually any safeguard, capable of--as a doomed doctor chillingly
describes it--"assimilating flesh on contact.
Snide comparisons to gelatin be damned, it's a concept with the most
devastating of potential consequences, not unlike the grey goo scenario
proposed by technological theorists fearful of
artificial intelligence run rampant.
'''

blob = TextBlob(text)
blob.tags           # [('The', 'DT'), ('titular', 'JJ'),
                    #  ('threat', 'NN'), ('of', 'IN'), ...]

blob.noun_phrases   # WordList(['titular threat', 'blob',
                    #            'ultimate movie monster',
                    #            'amoeba-like mass', ...])

for sentence in blob.sentences:
    print(sentence.sentiment.polarity)
# 0.060
# -0.341

TextBlob站在NLTK和pattern，并且两者都玩得很好

功能

  • 名词短语提取
  • 词性标注
  • 情绪分析
  • 分类(朴素贝叶斯、决策树)
  • 标记化(将文本拆分成单词和句子)
  • 词频和词频
  • 解析
  • N元语法
  • 词形变化(复数和单数)与词汇化
  • 拼写更正
  • 通过扩展添加新模型或语言
  • Wordnet集成

现在就去拿吧

$ pip install -U textblob
$ python -m textblob.download_corpora

示例

查看更多示例，请参阅Quickstart guide

文档

有关完整文档，请访问https://textblob.readthedocs.io/

要求

  • Python>=2.7或>=3.5

项目链接

许可证

麻省理工学院有执照。请参阅捆绑的LICENSE有关更多详细信息，请提交文件