Syllable-Pattern-Based Unknown-Morpheme Segmentation and Estimation for Hybrid Part-of-Speech Tagging of Korean
Bibliographic Data
| ID | 12156281 |
|---|---|
| Authors | Gary geunbae Lee (NLP Laboratory, Electrical and Computer Engineering Division, Pohang University of Science Technology (POSTECH), Pohang, 790-784, Korea., corresponding author), Jeong-Won Cha (Pohang University of Science and Technology), Jeongwon Cha (NLP Laboratory, Electrical and Computer Engineering Division, Pohang University of Science and Technology (POSTECH), Pohang, 790-784, Korea.), Jong-Hyeok Lee (0000-0002-9584-856X, NLP Laboratory, Electrical and Computer Engineering Division, Pohang University of Science and Technology (POSTECH), Pohang, 790-784, Korea.) |
| Year | 2002 |
| Volume | 28 |
| Issue | 1 |
| Pages | 53-70 |
| Publication date | 2002-03-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Computational Linguistics (JOURNAL) |
| Journal identifiers | ISSN: 0891-2017 • E-ISSN: 1530-9312 |
| Publisher | Association for Computational Linguistics (PUBLISHER • US) |
| DOI | 10.1162/089120102317341774 |
| OpenAlex | W2099227075 |
| Language | EN |
| Citations received | 2 |
| References cited | 2 |
Most errors in Korean morphological analysis and part-of-speech (POS) tagging are caused by unknown morphemes. This paper presents a syllable-pattern-based generalized unknown-morpheme-estimation method with POSTAG (POStech TAGger), 1 which is a statistical and rule-based hybrid POS tagging system. This method of guessing unknown morphemes is based on a combination of a morpheme pattern dictionary that encodes general lexical patterns of Korean morphemes with a posteriori syllable trigram estimation. The syllable trigrams help to calculate lexical probabilities of the unknown morphemes and are utilized to search for the best tagging result. This method can guess the POS tags of unknown morphemes regardless of their numbers and/or positions in an eojeol (a Korean spacing unit similar to an English word), which is not possible with other systems for tagging Korean. In a series of experiments using three different domain corpora, the system achieved a 97% tagging accuracy even though 10% of the morphemes in the test corpora were unknown. It also achieved very high coverage and accuracy of estimation for all classes of unknown morphemes. 1 The binary code of POSTAG is open to the public for research and evaluation purposes at http://nlp.postech.ac.kr/. Follow the link OpenResources→DownLoad
Morpheme · Natural language processing · Part of speech · Pattern recognition (psychology · Segmentation · Speech recognition · Syllable · Trigram · Word (group theory · Algorithms and Data Compression · Computer Science · Mathematics · Natural Language Processing Techniques · Topic Modeling · Artificial Intelligence
| Citations per year | 0.2 |
|---|---|
| Citation span | 2016 - 2024 (9) |
| Highly cited | No |