An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems

Abstract

The paper investigates error-entropy-minimization in adaptive systems training. We prove the equivalence between minimization of error's Renyi (1970) entropy of order /spl alpha/ and minimization of a Csiszar (1981) distance measure between the densities of desired and system outputs. A nonparametric estimator for Renyi's entropy is presented, and it is shown that the global minimum of this estimator is the same as the actual entropy. The performance of the error-entropy-minimization criterion is compared with mean-square-error-minimization in the short-term prediction of a chaotic time series and in nonlinear system identification.