Entropy, information, and computation

Abstract

The relation between entropy, information, and randomness is discussed. Algorithmic information theory is introduced and used to provide a fundamental definition of entropy. The relation between algorithmic entropy and the usual Shannon–Gibbs entropy is discussed.