Deep reinforcement learning for gearshift controllers in automatic transmissions

Abstract

Control design for gearshifts in modern automotive automatic transmissions constitutes a challenging, time consuming task performed by highly trained experts. This is due to the fact that a variety of non-linear and partially observable systems need to be actuated, such that a comfortable shifting behavior is achieved within an sufficiently low shifting time. The presented approach leverages deep reinforcement learning (DRL) to control gear shifts, outperforming current state of the art controller performance. This requires formulating the shifting task as a Markov decision process by designing suitable action and observation spaces as well as a meaningful reward function. Due to the sample complexity of DRL methods, the control agents are trained in simulation and are subsequently transferred to a real transmission on a test bench. To successfully transfer DRL agents from simulation to reality, methods such as domain randomization and domain adaption leveraging evolutionary optimization are applied. To the best of the authors’ knowledge, this work is the first to successfully apply DRL for the closed loop control of a real world automotive automatic transmission of realistic complexity.