Description of the Kent Ridge Digital Labs System Used for MUC-7

Abstract

We aim to build a single simple framework for tasks in text information extraction, for which, to a certain extent, the required information can be resolved locally. Our system is statistics-based. As usual, language model is built from training corpus. This is the so-called learning process. Much e ort has been spent to absorb domain knowledge in the language model in a systematic and generic way, because the system is designed not for one particular task, but for general local information extraction. For the information extraction part (tagging), the system consists of the following modules: Sentence segmentor and tokenizer. This module accepts a stream of characters as input, and transforms it into a sequence of sentences and tokens. The way of tokenization can vary with di erent tasks and domains. For example, most English text is tokenized in the same way, while tokenization in Chinese itself is a research topic. Text analyzer. This module provides analysis necessary for the particular task, be it semantic, syntactic, orthographic, etc. This same analyzer is also applied in the learning process. Hypothesis generator. The possibilities for each word (token) are determined. Rules