In-memory Reinforcement Learning with Moderately-Stochastic Conductance Switching of Ferroelectric Tunnel Junctions

Abstract

Building compact and efficient reinforcement learning (RL) systems for mobile deployment requires departure from the von-Neumann computing architecture and embracing novel in-memory computing, and local learning paradigms. We exploit nano-scale ferroelectric tunnel junction (FTJ) memristors with inherent analogue stochastic switching arranged in selector-less crossbars to demonstrate an analogue in-memory RL system, which, via a hardware-friendly algorithm, is capable of learning behavior policies. We show that commonly undesirable stochastic conductance switching is actually, in moderation, a beneficial property which promotes policy finding via a process akin to random search. We experimentally demonstrate path-finding based on reinforcement, and solve a standard control problem of balancing a pole on a cart via simulation, outperforming similar deterministic RL systems.