A FORMULA TO CALCULATE PRUNING THRESHOLD FOR THE PART-OF-SPEECH TAGGING PROBLEM

Authors

  • Nguyen Chi Hieu Industrial University of Ho Chi Minh City, 12 Nguyen Van Bao, Ward 4, Go Vap District, Ho Chi Minh City

DOI:

https://doi.org/10.15625/2525-2518/54/3A/11959

Keywords:

Hidden Markov model, Part-of-speech tagging, Viterbi algorithm, Beam search.

Abstract

The exact tagging of the words in the texts is a very important task in the natural language processing. It can support parsing the text, contribute to the solution of the polysemous word, and help to access a semantic information, etc. One of crucial factors in the POS (Part-of-Speech) tagging approaches based on the statistical method is the processing time. In this paper, we propose an approach to calculate the pruning threshold, which can apply into the Viterbi algorithm of Hidden Markov model for tagging the texts in the natural language processing. Experiment on the 1.000.000 words on the tag of the Wall Street Journal corpus showed that our proposed solution is satisfactory.

Downloads

Download data is not yet available.

Downloads

Published

2018-03-20

How to Cite

Hieu, N. C. (2018). A FORMULA TO CALCULATE PRUNING THRESHOLD FOR THE PART-OF-SPEECH TAGGING PROBLEM. Vietnam Journal of Science and Technology, 54(3A), 64. https://doi.org/10.15625/2525-2518/54/3A/11959

Issue

Section

Articles