Stability of analog neural networks with delay

TLDR

Continuous‑time analog neural networks with symmetric connections converge to fixed points when neurons respond instantaneously, but small time delays can induce sustained oscillations, a phenomenon relevant to hardware implementations where amplifier switching delays are comparable to network relaxation times. The study analyzes continuous‑time analog neural networks with delay to identify a critical delay threshold that triggers oscillation in symmetrically connected networks. The authors present two stability analyses for low and high neuron gain and compare theoretical predictions with numerical simulations and experiments on an eight‑neuron electronic network with controllable delay. The results provide design criteria, showing that certain connection topologies can oscillate with delays far below the relaxation time, associative memory networks tolerate long delays, all‑inhibitory networks are most oscillation‑prone with a critical delay N times smaller than the relaxation time, and theoretical predictions agree with simulations and experiments.

Abstract

Continuous-time analog neural networks with symmetric connections will always converge to fixed points when the neurons have infinitely fast response, but can oscillate when a small time delay is present. Sustained oscillation resulting from time delay is relevant to hardware implementations of neural networks where delay due to the finite switching speed of amplifiers can be appreciable compared to the network relaxation time. We analyze the dynamics of continuous-time analog networks with delay, and show that there is a critical delay above which a symmetrically connected network will oscillate. Two different stability analyses are presented for low and high neuron gain. The results are useful as design criteria for building fast but stable electronic networks. We find that for some connection topologies, a delay much smaller than the relaxation time can lead to oscillation, whereas for other topologies, including associative memory networks, even long delays will not produce oscillation. The most oscillation-prone network configuration is the all-inhibitory network; in this configuration, the critical delay for oscillation is smaller than the network relaxation time by a factor of N, the size of the network. Theoretical results are compared with numerical simulations and with experiments performed on a small (eight neurons) electronic network with controllable delay.