THE CO-INFORMATION LATTICE

Abstract

In 1955, McGill published a multivariate generalisation of Shannon’s mutual information. Algorithms such as Independent Component Analysis use a different generalisation, the redundancy, ormulti-information [13]. McGill’s concept expresses the information shared by all of K random variables, while the multi-information expresses the information shared by any two or more of them. Partly to avoid confusion with the multiinformation, I call his concept here the co-information. Co-informations, oddly, can be negative. They form a partially ordered set, or lattice, as do the entropies. Entropies and co-informations are simply and symmetrically related by Möbius inversion [12]. The co-information lattice sheds light on the problem of approximating