Toward a modern theory of adaptive networks: Expectation and prediction.

TLDR

Adaptive neural network theories often use neuronlike elements that mimic associative conditioning, yet they have largely ignored the predictive nature of classical conditioning; the present model aligns with Rescorla‑Wagner predictions and recent synaptic physiology findings. The study develops an adaptive element that better reflects animal learning theory than conventional adaptive network elements. The element learns to raise its response rate in anticipation of heightened stimulation, generating a conditioned response prior to the unconditioned stimulus. Computer simulations demonstrate that the element becomes sensitive to the most reliable, nonredundant, and earliest predictors of reinforcement and that it resolves stability and saturation issues common in network simulations.

Abstract

Many adaptive neural network theories are based on neuronlike adaptive elements that can behave as single unit analogs of associative conditioning. In this article we develop a similar adaptive element, but one which is more closely in accord with the facts of animal learning theory than elements commonly studied in adaptive network research. We suggest that an essential feature of classical conditioning that has been largely overlooked by adaptive network theorists is its predictive nature. The adaptive element we present learns to increase its response rate in anticipation of increased stimulation, producing a conditioned response before the occurrence of the unconditioned stimulus. The element also is in strong agreement with the behavioral data regarding the effects of stimulus context, since it is a temporally refined extension of the Rescorla-Wagner model. We show by computer simulation that the element becomes sensitive to the most reliable, nonredundant, and earliest predictors of reinforcement . We also point out that the model solves many of the stability and saturation problems encountered in network simulations. Finally, we discuss our model in light of recent advances in the physiology and biochemistry of synaptic mechanisms.