Approximation–Theoretic Derivation of Logarithmic Entropy Principles for Inverse Problems and Unique Extension of the Maximum Entropy Method to Incorporate Prior Knowledge

Abstract

The minimum distance approach for reconstructing a positive function based on knowledge of finitely many linear functional values is examined. Two important classes of directed distances for signal processing and statistical inference are discussed. By imposing conditions analogous to those satisfied by linear projections in Hilbert space, two logarithmic entropy principles are derived. One of these involves the Itakura–Saito distortion measure of communication theory and uniquely extends Burg's maximum entropy method to incorporate prior knowledge. The other uses the Kullback–Leibler distance of statistics.