What is Part of Speech Tagging in Natural Language Processing?

June 1, 2018 Author: munishmishra04_3od47tgp
Print Friendly, PDF & Email

With the advancement of technology, the demand of Natural Language Processing (NLP) is also increasing and it becomes very important to find out correct information from collection of huge data only on the basis of queries and keywords. Sometimes user tries to search data with help of query and get unimportant or irrelevant data instead of correct data. The aim of Natural Language Processing is to facilitate the interaction between human and machine. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation. The solution for language understanding is Part of Speech tagging. The basic system is human-computer interaction, which allows user to interact with computer using their everyday languages.

Basic Overview of Part of Speech Tagging





A Part of Speech Tagger (POS Tagger) is a piece of software that reads text in some language and assigns parts of speech to each word (and other token), such as noun, verb, adjective, etc., although generally computational applications use more fine-grained POS tags like ‘noun-plural’.

Tagging

Automatic assignment of descriptors to the given tokens is called Tagging. The descriptor is called tag. The tag may indicate one of the parts-of-speech, semantic information, and so on. So that tagging a kind of classification.

POS

The process of assigning one of the parts of speech to the given word is called Parts of Speech tagging. It is commonly referred to as Part of Speech tagging. Parts of speech include nouns, verbs, adverbs, adjectives, pronouns, conjunction and their sub-categories.

Example:

Word: Paper, Tag: Noun

Word: Go, Tag: Verb

Word: Famous, Tag: Adjective

Note that some words can have more than one tag associated with. For example, chair can be noun or verb depending on the context.

Parts of Speech Tagger

Parts of Speech tagger or POS tagger is a program that does this job. Taggers use several kinds of information: dictionaries, lexicons, rules, and so on. Dictionaries have category or categories of a particular word. That is a word may belong to more than one category. For example, run is both noun and verb. Taggers use probabilistic information to solve this ambiguity .

There are mainly two types of taggers: rule-based and stochastic. Rule-based taggers use hand-written rules to distinguish the tag ambiguity. Stochastic taggers are either HMM based, choosing the tag sequence which maximizes the product of word likelihood and tag sequence probability, or cue-based, using decision trees or maximum entropy models to combine probabilistic features.
Ideally a typical tagger should be robust, efficient, accurate, tunable and reusable. In reality taggers either definitely identify the tag for the given word or make the best guess based on the available information. As the natural language is complex it is sometimes difficult for the taggers to make accurate decisions about tags. So that occasional errors in tagging is not taken as a major roadblock to research.

Tag-set

Tagset is the set of tags from which the tagger is supposed to choose to attach to the relevant word. Every tagger will be given a standard tagset. The tagset may be coarse such as N (Noun), V(Verb), ADJ(Adjective), ADV(Adverb), PREP(Preposition), CONJ(Conjunction) or fine-grained such as NNOM(Noun-Nominative), NSOC(Noun-Sociative), VFIN(Verb Finite),VNFIN(Verb Nonfinite) and so on. Most of the taggers use only fine grained tagset [42].

Steps of POS Tagger

  1. Tokenization: The given text is divided into tokens so that they can be used for further analysis. The tokens may be words, punctuation marks, and utterance boundaries.
  2. Ambiguity look-up: This is to use lexicon and a guessor for unknown words. While lexicon provides list of word forms and their likely parts of speech, guessors analyze unknown tokens. Compiler or interpreter, lexicon and guessor make what is known as lexical analyzer.
  3. Ambiguity Resolution: This is also called disambiguation. Disambiguation is based on information about word such as the probability of the word. For example, power is more likely used as noun than as verb. Disambiguation is also based on contextual information or word/tag sequences. For example, the model might prefer noun analyses over verb analyses if the preceding word is a preposition or article. Disambiguation is the most difficult problem in tagging.

POS Tagging Process





Process of Part Of Speech Tagging: Read the input sentence. Then tokenize the sentence into words. After tokenization, Suffix analysis and prefix analysis is also used for correctly tag each word of sentence. Then use one of the tagging methods to tag each word of sentence of corpus as noun, verb, conjunction, number tag etc. The output is tagged sentence. Then after evaluate the accuracy of output. Part Of Speech tagging has got much significance in field of computational linguistics which uses algorithms to associate discrete terms, as well as its hidden parts of speech with respect to a set of descriptive tags. Part of speech tagging is used to introduce the relationship of one word with its previous word as well as its next word.

POS Tagging Process

Figure: Process of POS Tagging

References





[1] Robin, “Parts-of-Speech Tagging”, available online at: http://language.worldofcomputing.net/pos-tagging/parts-of-speech-tagging.html

[2] S. M. Mohammad, S. Kiritchenko, and X. Zhu, “NRC-canada: Building the state-of-the-art in sentiment analysis of tweets”, in Proceedings of the seventh international workshop on Semantic Evaluation Exercises, 2013, pp. 321–327.

[3] “Software > Stanford Log-linear Part-Of-Speech Tagger”, available online at: https://nlp.stanford.edu/software/tagger.shtml

[4]  Nidhi Adhvaryu and Prem Balani, “Survey: Part-Of-Speech Tagging in NLP”, International Journal of Research in Advent Technology, ICATEST 2015”, 08 March 2015.

One Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Insert math as
Block
Inline
Additional settings
Formula color
Text color
#333333
Type math using LaTeX
Preview
\({}\)
Nothing to preview
Insert