IEEE Robotics and Automation Letters · 2023 · 25 citations · 22 references
ComputingComputer ScienceComputer VisionMachine LearningMotor ControlDeep LearningReinforcement LearningSelf-supervised LearningSelf-supervised Robot LearningObject-centric LearningSelf-supervised PretrainingSelf-supervised Reinforcement LearningSelf-supervised Representation LearningVision Self-supervised LearningVoxel-based ModelingVoxel-based ReconstructionVoxel-based Representation
A prominent approach to visual Reinforcement Learning (RL) is to learn an internal state representation using self-supervised methods, which has the potential benefit of improved sample-efficiency and generalization through additional learning signal and inductive biases. However, while the real world is inherently 3D, prior efforts have largely been focused on leveraging 2D computer vision techniques as auxiliary self-supervision. In this work, we present a unified framework for self-supervised learning of 3D representations for motor control. Our proposed framework consists of two phases: a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pretraining</i> phase where a deep voxel-based 3D autoencoder is pretrained on a large object-centric dataset, and a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">finetuning</i> phase where the representation is jointly finetuned together with RL on in-domain data. We empirically show that our method enjoys improved sample efficiency compared to 2D representation learning methods. Additionally, our learned policies transfer zero-shot to a real robot setup with only approximate geometric correspondence, and successfully solve motor control tasks that involve grasping and lifting from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a single, uncalibrated RGB camera</i> .
22
Improved Baselines with Momentum Contrastive Learning
Xinlei Chen, Haoqi Fan, Ross Girshick et al. · arXiv (Cornell University) · 2020 · 2.1K citations · Full text
Learning dexterous in-hand manipulation
OpenAI Marcin Andrychowicz, Bowen Baker, Maciek Chociej et al. · The International Journal of Robotics Research · 2019 · 1.6K citations · Full text
End-to-End Training of Deep Visuomotor Policies
Sergey Levine, Chelsea Finn, Trevor Darrell et al. · arXiv (Cornell University) · 2015 · 1.4K citations · Full text