DeepLabStream: Closing the loop using deep learning-based markerless, real-time posture detection

Abstract

Abstract In general, animal behavior can be described as the neuronal-driven sequence of reoccurring postures through time. Current technologies enable offline pose estimation with high spatio-temporal resolution, however to understand complex behaviors, it is necessary to correlate the behavior with neuronal activity in real-time. Here we present DeepLabStream, a highly versatile, closed-loop solution for freely moving mice that can autonomously conduct behavioral experiments ranging from behavior-based learning tasks to posture-dependent optogenetic stimulation. DeepLabStream has a temporal resolution in the millisecond range, can operate with multiple devices and can be easily tailored to a wide range of species and experimental designs. We employ DeepLabStream to autonomously run a second-order olfactory conditioning task for freely moving mice and to deliver optogenetic stimuli based on mouse head-direction.