Supervised latent semantic indexing using adaptive sprinkling

Abstract

Latent Semantic Indexing (LSI) has been shown to be effective in recovering from synonymy and pol-ysemy in text retrieval applications. However, since LSI ignores class labels of training documents, LSI generated representations are not as effective in classification tasks. To address this limitation, a process called ‘sprinkling ’ is presented. Sprinkling is a simple extension of LSI based on augmenting the set of features using additional terms that en-code class knowledge. However, a limitation of sprinkling is that it treats all classes (and classi-fiers) in the same way. To overcome this, we pro-pose a more principled extension called Adaptive Sprinkling (AS). AS leverages confusion matrices to emphasise the differences between those classes which are hard to separate. The method is tested on diverse classification tasks, including those where classes share ordinal or hierarchical relationships. These experiments reveal that AS can significantly enhance the performance of instance-based tech-niques (kNN) to make them competitive with the state-of-the-art SVM classifier. The revised repre-sentations generated by AS also have a favourable impact on SVM performance. 1