Linear Least-Squares Smoothing and Prediction, with Applications

Abstract

This paper describes the calculation of smoothing and prediction operators of the linear least-squares sort using techniques derived from a circuit theory point of view. The techniques are developed explicitly for time series which are continuous and statistically stationary. Other situations are explored more briefly, however, in which the time series are either discrete or statistically nonstationary. For the most part, functions of time are replaced by functions of frequency, representing their transforms. Mathematical complications are avoided by restricting statistical ensembles to those which have rational power spectra. In practice, actual spectra can be approximated sufficiently well by rational spectra, and the simplified methods are sufficiently general for engineering applications of many different sorts. Both finite and semi-infinite smoothing intervals are permitted, with or without constraints of various sorts. The assumption of rational spectra does not apply directly to nonstationary time series, but it may be replaced by a closely analogous restriction which does apply. Then there are nonstationary operations which are closely analogous to the stationary operations, developed for stationary systems. A brief examination of these analogies is of interest, even though the nonstationary operations are usually too complicated for engineering purposes. The general techniques are developed in terms of specific problems, chosen for purposes of exposition and because of their engineering interest.