bioRxiv (Cold Spring Harbor Laboratory) · 2020 · 16 citations · 58 references
ComputingZoologyAnimal BehaviorBehavioral NeuroscienceComputer Vision3D ReconstructionMulti-view GeometryPose EstimationScene Reconstruction2D Pose Estimation3D Pose EstimationAnimal Pose EstimationCalibration ProceduresCalibration ProtocolsStereotyped Behaviors
Abstract Markerless 3D pose estimation has become an indispensable tool for kinematic studies of laboratory animals. Most current methods recover 3D pose by multi-view triangulation of deep network-based 2D pose estimates. However, triangulation requires multiple, synchronized cameras and elaborate calibration protocols that hinder its widespread adoption in laboratory studies. Here, we describe LiftPose3D, a deep network-based method that overcomes these barriers by reconstructing 3D poses from a single 2D camera view. We illustrate LiftPose3D’s versatility by applying it to multiple experimental systems using flies, mice, rats, and macaque monkeys and in circumstances where 3D triangulation is impractical or impossible. Our framework achieves accurate lifting for stereotyped and non-stereotyped behaviors from different camera angles. Thus, LiftPose3D permits high-quality 3D pose estimation in the absence of complex camera arrays, tedious calibration procedures, and despite occluded body parts in freely behaving animals.
58
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Zhe Cao, Tomas Simon, Shih-En Wei et al. · 2017 · 7.3K citations