Evidentiality for Text Trustworthiness Detection

Abstract

Evidentiality is the linguistic representation of the nature of evidence for a statement. In other words, it is the linguistically encoded evidence for the trustworthiness of a statement. In this paper, we aim to explore how linguistically encoded information of evidentiality can contribute to the prediction of trustworthiness in natural language processing (NLP). We propose to incorporate evidentiality into a framework of machine learning based text classification. We first construct a taxonomy of evidentials. Then experiments involving collaborative question answering (CQA) are designed and implemented using this taxonomy. The experimental results confirm that evidentiality is an important clue for text trustworthiness detection. With the binarized vector setting, evidential based text representation model has considerably performaned better than both the bag-of-word model and the content word based model. Most crucially, we show that the best trustworthiness detection result is achieved when evidentiality is incorporated in a linguistically sophisticated model where their meanings are interpreted in both semantic and pragmatic terms. 1