Robust Pose Estimation via Hybrid Point and Twin Line Reprojection for RGB-D Vision Navigation

Abstract

Pose estimation is a crucial technique to achieve robot localization using natural features for an RGB-D vision navigation system. However, it still suffers from the impoverishment of robustness and accuracy in complex workspaces. A robust real-time pose estimation approach using hybrid point and twin line reprojection is proposed to allow accurate localization for a vision-based robot under various challenging scenarios including low-texture, low-structure, occlusion, and viewpoint and illumination changes. Firstly, an improved adaptive threshold (IAT)-based oriented fast and rotated Brief (ORB) feature detection method is developed to extract sufficient IAT-ORB point features, which are matched by a multi-stage KNN-RTC-CS-RANSAC (KRCR) refinement feature matching strategy to generate reliable point matches. Then a robust EDLines line feature matching method is developed by integrating dynamic matching (DM) and median absolute deviation (MAD) to obtain fine line matches. Furthermore, the hybrid point reprojection errors are built from the fusion of 3D-2D and 3D-3D point pairs, and the twin line reprojection errors are constructed by integrating virtual right-eye lines generated by depth measurement, which both are incorporated into a novel unified errors optimization model to achieve optimal motion estimation by minimizing the reprojection errors of points and lines concurrently. Finally, the extensive experiments verify the robustness, accuracy, and real-time performance of the proposed approach on the public datasets and our real indoor scenarios. The quantitative results demonstrate that our proposed approach gains more accurate and stable localization than other state-of-the-art methods under challenging environments.