The importance of proper weighting methods

Abstract

The importance of good weighting methods in information retrieval --- methods that stress the most useful features of a document or query representative --- is examined. Evidence is presented that good weighting methods are more important than the feature selection process and it is suggested that the two need to go hand-in-hand in order to be effective. The paper concludes with a method for learning a good weight for a term based upon the characteristics of that term.