On the rationale of maximum-entropy methods

TLDR

The authors aim to examine the relationships between maximum‑entropy and other spectral analysis techniques. They argue that these methods are complementary rather than conflicting, each suited to different problem contexts. They conclude that orthodox sampling methods are best when noise models are known but priors are lacking; MAXENT excels when priors are available but noise is absent; the full Bayesian approach unifies both; AR models are a special case of MAXENT yet ubiquitous; and empirical methods like Blackman‑Tukey are useful for exploratory analysis but insufficient for fully automated quantitative modeling.

Abstract

We discuss the relations between maximum-entropy (MAXENT) and other methods of spectral analysis such as the Schuster, Blackman-Tukey, maximum-likelihood, Bayesian, and Autoregressive (AR, ARMA, or ARIMA) models, emphasizing that they are not in conflict, but rather are appropriate in different problems. We conclude that: 1) "Orthodox" sampling theory methods are useful in problems where we have a known model (sampling distribution) for the properties of the noise, but no appreciable prior information about the quantities being estimated. 2) MAXENT is optimal in problems where we have prior information about multiplicities, but no noise. 3) The full Bayesian solution includes both of these as special cases and is needed in problems where we have both prior information and noise. 4) AR models are in one sense a special case of MAXENT, but in another sense they are ubiquitous in all spectral analysis problems with discrete time series. 5) Empirical methods such as Blackman-Tukey, which do not invoke even a likelihood function, are useful in the preliminary, exploratory phase of a problem where our knowledge is sufficient to permit intuitive judgments about how to organize a calculation (smoothing, decimation, windows, prewhitening, padding with zeroes, etc.) but insufficient to set up a quantitative model which would do the proper things for us automatically and optimally.