IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality

Abstract

Robotic assembly is a longstanding challenge, requiring contact-rich interaction and high precision and accuracy.Many applications also require adaptivity to diverse parts, poses, and environments, as well as low cycle times.In other areas of robotics, simulation is a powerful tool to develop algorithms, generate datasets, and train agents.However, simulation has had a more limited impact on assembly.We present IndustReal, a set of algorithms, systems, and tools that solve assembly tasks in simulation with reinforcement learning (RL) and successfully achieve policy transfer to the real world.Specifically, we propose 1) simulation-aware policy updates, 2) signed-distancefield rewards, and 3) sampling-based curricula for robotic RL agents.We use these algorithms to enable robots to solve contactrich pick, place, and insertion tasks in simulation.We then propose 4) a policy-level action integrator to minimize error at policy deployment time.We build and demonstrate a real-world robotic assembly system that uses the trained policies and action integrator to achieve repeatable performance in the real world.Finally, we present hardware and software tools that allow other researchers to reproduce our system and results.For videos and additional details, please see our project website.the presence of unmodeled dynamics (e.g., friction).• Benchmarks: We solve several challenging tasks proposed in Factory [48] (pick, place, and insertion tasks for pegs-and-holes and gear assemblies in simulation) with success rates of 82-99%.We provide careful evaluations over 265k simulated trials to show the utility of SAPU, SDF-based rewards, and SBC for solving these tasks.• Systems: We design and demonstrate a real-world system that can perform sim-to-real transfer of our simulationtrained policies, with success rates of 83-99% over 600 trials.We provide careful evaluations to show the utility of PLAI.To our knowledge, this is the first system for sim-to-real of all phases of the assembly problem: from detection, to grasping, to part alignment, to insertion.Our system uses commonly-used robotics hardware and requires no real-world policy adaptation phase.Our secondary contributions are the following:• Hardware: We present IndustRealKit, which contains CAD models for all parts designed for our setup, as well as a list of all purchased parts.The CAD models can all be printed on a desktop 3D printer.IndustRealKit allows the research community to easily replicate our experimental hardware and benchmark their performance.• Software: We present IndustRealLib, a lightweight Python library that allows users to easily deploy policies trained in NVIDIA Isaac Gym [43] onto a real-world Franka Emika Panda robot [17].The library also contains code to assist with policy training.IndustRealLib allows the research community to reproduce our robot behaviors.We aim for IndustReal to provide algorithms, benchmark results, and a reproducible system that serve as a path forward for sim-to-real transfer on contact-rich assembly tasks. II. RELATED WORKWe divide prior work on robotic assembly into three categories: 1) classical approaches leveraging analytical methods [70,45], 2) learning-based approaches leveraging realworld data or experience [75], and 3) RL-based sim-to-real approaches leveraging robotics simulators.We defer a review of ( 1) and (2) to Appendix A and focus on (3). A. Sim-to-Real Transfer for AssemblyOver the past few years, there have been a number of impressive efforts in sim-to-real for assembly.These efforts have primarily used MuJoCo [52,11,20,67,79,27] or PyBullet [38,56,54]; have used PPO [56,58,20,27] or DDPG [38,6]; and have aimed to solve peg-in-hole [38,7,54,52,58,11,67,79,27] or NIST-style tasks [7,52,11,79].However, several of these studies use large clearances (e.g., ≥ 1 mm) and/or large parts in simulation and/or the real world.Furthermore, almost all use force/torque (F/T) sensors to collect observations and/or set thresholds.Most require human demonstrations [38,6,11], a baseline motion plan [56,58,27], and/or fine-tuning in the real-world [7,52].Finally, all but one [52] focus only on insertion and assume the object is pre-grasped; however, [52] also uses specialized