The entropy of ordered sequences and order statistics

Abstract

The entropy of a sequence of random variables under order restrictions is examined. A theorem that shows the amount of entropy reduction when the sequence is ordered is presented. Upper and lower bounds to the entropy reduction and conditions under which they are achieved are derived. Some interesting properties of the entropy of the individual order statistics are also presented. It is shown that the difference between the average entropy of the individual order statistics and the entropy of a member of the original independent identically distributed (IID) population is a constant, regardless of the original distribution. Finally, the entropies of the individual order statistics are found to be symmetric about the median when the probability density function (PDF) of the original IID sequence is symmetric about its mean.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>