Similarity of attributes by external probes

Abstract

In data mining, similarity between attributes is one of the central notions. Such a notion can be used to build attribute hierarchies etc. Similarity metrics can be user-defined, but an important problem is defining similarity on the basis of data. Several methods based on statistical techniques exist, but for defining the similarity between A and B they typically consider only the values of A and B, not the other attributes. We describe how a similarity or distance between attributes can be defined by considering the other attributes. The basic idea is that in a 0/1 relation r, two attributes A and B are similar if the subrelations oe A=1 (r) and oe B=1 (r) are similar. Similarity between these relations is defined by considering the marginal frequencies of a selected subset of other attributes. Thus for example in a market basket database two products A and B would be deemed similar if the customers buying A and B have similar buying behavior with respect to the other products. We ...