ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM

Unknown author(s)

IEEE Transactions on Robotics · 2021 · 3.8K citations · 59 references

DOIFull text

Open access

Concepts

TL;DR

This paper introduces ORB‑SLAM3, the first system capable of visual, visual‑inertial, and multi‑map SLAM across monocular, stereo, and RGB‑D cameras with pin‑hole and fisheye lenses. It achieves this through a feature‑based, MAP‑estimated visual‑inertial SLAM, a novel multi‑map system with improved place recognition, and bundle adjustment that incorporates co‑visible keyframes across time, all implemented in an open‑source library. ORB‑SLAM3 runs in real‑time, is 2–5× more accurate than prior methods, remains robust in indoor and outdoor environments, can recover from lost tracks by merging new maps, reuses all past information, and attains centimeter‑level accuracy on EuRoC and millimeter‑level accuracy in handheld AR/VR scenarios.

Abstract

This paper presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. The first main novelty is a feature-based tightly-integrated visual-inertial SLAM system that fully relies on Maximum-a-Posteriori (MAP) estimation, even during the IMU initialization phase. The result is a system that operates robustly in real-time, in small and large, indoor and outdoor environments, and is 2 to 5 times more accurate than previous approaches. The second main novelty is a multiple map system that relies on a new place recognition method with improved recall. Thanks to it, ORB-SLAM3 is able to survive to long periods of poor visual information: when it gets lost, it starts a new map that will be seamlessly merged with previous maps when revisiting mapped areas. Compared with visual odometry systems that only use information from the last few seconds, ORB-SLAM3 is the first system able to reuse in all the algorithm stages all previous information. This allows to include in bundle adjustment co-visible keyframes, that provide high parallax observations boosting accuracy, even if they are widely separated in time or if they come from a previous mapping session. Our experiments show that, in all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature, and significantly more accurate. Notably, our stereo-inertial SLAM achieves an average accuracy of 3.6 cm on the EuRoC drone and 9 mm under quick hand-held motions in the room of TUM-VI dataset, a setting representative of AR/VR scenarios. For the benefit of the community we make public the source code.

References

59