Learning entropy for novelty detection a cognitive approach for adaptive filters

Abstract

This paper recalls the practical calculation of Learning Entropy (LE) for novelty detection, extends it for various gradient techniques and discusses its use for multivariate dynamical systems with ability of distinguishing between data perturbations or system-function perturbations. LE has been recently introduced for novelty detection in time series via supervised incremental learning of polynomial filters, i.e. higher-order neural units (HONU). This paper demonstrates LE also on enhanced gradient descent adaptation techniques that are adopted and summarized for HONU. As an aside, LE is proposed as a new performance index of adaptive filters. Then, we discuss Principal Component Analysis and Kernel PCA for HONU as a potential method to suppress detection of data-measurement perturbations and to enforce LE for system-perturbation novelties.