Feature-based Thai unknown word boundary identification using Winnow

Abstract

This paper addresses the problem of Thai unknown word boundary identification. Unknown words are becoming the main problem in many tasks of natural language processing such as word segmentation information retrieval and part of speech tagging, etc.. In Thai, as words are written consecutively without delimiters, finding an unknown word boundary is difficult. We proposed a feature-based approach to identify Thai unknown word boundary. A feature can be anything that tests for specific information in context around the target unknown words. To automatically extract features from a training corpus, we used a machine learning algorithm, namely Winnow.