Frontiers in Neurorobotics · 2019 · 176 citations · 31 references
ComputingComputer SciencePhysicsArtificial IntelligenceOptimizationRoboticsDeep LearningDerivative-free OptimizationNeural NetworksOptimization For Machine LearningReinforcement LearningDeep Learning On GpusEvolutionary AlgorithmsFinite Difference ApproximationOptimization AlgorithmsReinforcement Learning For Robotics
Robotics controller optimization often treats robots as black boxes, forcing reliance on derivative‑free methods; gradient‑based approaches require small models or finite‑difference Jacobians, which become prohibitively expensive as parameter counts grow. The authors propose a modern physics engine that can differentiate control parameters to enable gradient‑based optimization of robotic controllers. This engine is implemented for both CPU and GPU, allowing efficient back‑propagation through simulated dynamics. The engine accelerates optimization even on small problems and marks a major step for deep learning in robotics by opening new possibilities for hardware and software optimization.
An important field in robotics is the optimization of controllers. Currently, robots are often treated as a black box in this optimization process, which is the reason why derivative-free optimization methods such as evolutionary algorithms or reinforcement learning are omnipresent. When gradient-based methods are used, models are kept small or rely on finite difference approximations for the Jacobian. This method quickly grows expensive with increasing numbers of parameters, such as found in deep learning. We propose the implementation of a modern physics engine, which can differentiate control parameters. This engine is implemented for both CPU and GPU. Firstly, this paper shows how such an engine speeds up the optimization process, even for small problems. Furthermore, it explains why this is an alternative approach to deep Q-learning, for using deep learning in robotics. Finally, we argue that this is a big step for deep learning in robotics, as it opens up new possibilities to optimize robots, both in hardware and software.
31
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Advances in Neural Information Processing Systems 28
Peter Flach, Meelis Kull · 2015 · 14.7K citations
Automatic differentiation in PyTorch
Adam Paszke, Sam Gross, Soumith Chintala et al. · 2017 · 11.1K citations
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Martı́n Abadi, Ashish Agarwal, Paul Barham et al. · arXiv (Cornell University) · 2016 · 9.8K citations · Full text
MuJoCo: A physics engine for model-based control
Emanuel Todorov, Tom Erez, Yuval Tassa · 2012 · 4.4K citations
Engineering, Mechanical Engineering, Robotics Engineering +17