Verification and validation of language processing systems

Abstract

If Natural Language Processing (NLP) systems are viewed as intelligent systems then we should be able to make use of verification and validation (V&V) approaches and methods that have been developed in the intelligent systems community. This paper addresses language engineering infrastructure issues by considering whether standard V&V methods are fundamentally different than the evaluation practices commonly used for NLP systems, and proposes practical approaches for applying V&V in the context of language processing systems. We argue that evaluation, as it is performed in the NL community, can be improved by supplementing it with methods from the V&V community.