IEEE Transactions on Circuits and Systems I Regular Papers · 2022 · 24 citations · 26 references
ComputingComputer ScienceRobotics EngineeringComputer VisionMobile ComputingMobile RoboticsAugmented RealityAugmented Reality HardwareDeep LearningDigital Image ProcessingHardware AccelerationImage ProcessingConvolutional Neural NetworksEmbedded Hardware Co-designKeypoint DetectionKeypoint LocalizationMobile Augmented RealityMobile Image ProcessingNon-maximum SuppressionVisual Simultaneous Localization
Keypoint extraction is a key technique for Visual Simultaneous Localization and Mapping (VSLAM). Recently, Convolutional Neural Network (CNN) has been used in the keypoint extraction for improving the accuracy. As one of the state-of-the-art CNN based keypoint extraction techniques, the SuperPoint ranked top in the CVPR2020 image matching challenge. However, the use of complex CNN makes it difficult to meet the real-time performance on a mobile platform with limited resource such as mobile robots and wearable Augmented Reality (AR) devices. In this work, based on the SuperPoint, we proposed an FPGA-based real-time keypoint extraction hardware accelerator through algorithm-hardware co-design for mobile VSLAM applications, which is named as MobileSP. Several algorithm and hardware level design techniques have been proposed to reduce the computation and improve the processing speed while maintaining high accuracy, including a partially shared detection & description encoding architecture, a pre-sorting based Non-Maximum Suppression (NMS) engine and a software-hardware hybrid pipeline computing technique. The design has been implemented and evaluated on a ZCU104 FPGA board. It achieves real-time performance of 42 fps with low Absolute Trajectory Error (ATE) of 1.82 cm simultaneously, outperforming several state-of-the-art designs.
26
A Combined Corner and Edge Detector
Chris Harris, Matthew J. Stephens · 1988 · 12.5K citations
Backpropagation Applied to Handwritten Zip Code Recognition
Yann LeCun, Bernhard E. Boser, J. S. Denker et al. · Neural Computation · 1989 · 11.9K citations
ORB: An efficient alternative to SIFT or SURF
Ethan Rublee, Vincent Rabaud, Kurt Konolige et al. · 2011 · 10.4K citations