Adaptive information filtering

Abstract

The task of information filtering is to classify documents from a stream as either relevant or non-relevant according to a particular user interest with the objective to reduce information load. When using an information filter in an environment that is changing with time, methods for adapting the filter should be considered in order to retain classification accuracy. We favor a methodology that attempts to detect changes and adapts the information filter only if inevitable in order to minimize the amount of user feedback for providing new training data. Yet, detecting changes may require costly user feedback as well. This paper describes two methods for detecting changes without user feedback. The first method is based on evaluating an expected error rate, while the second one observes the fraction of classification decisions made with a confidence below a given threshold. Further, a heuristics for automatically determining this threshold is suggested and the performance of this approach is experimentally explored as a function of the threshold parameter. Some empirical results show that both methods work well in a simulated change scenario with real world data.