A 42fps full-HD ORB feature extraction accelerator with reduced memory overhead

Abstract

This paper proposes a hardware accelerator of Oriented FAST and Rotated BRIEF (ORB) algorithm with its full implementation. The accelerator is optimized in computational logics and memory organizations to achieve real-time performance and good matching accuracy. The architecture consists of a loose-coupled pipeline, in which keypoints are detected in scale-level parallel while descriptors are built in task-level parallel. In this way, the workload in different pipeline stages are balanced. Because of the loose-coupled architecture employed, large amounts of image pyramid data are offloaded to external memory and fetched again. To relieve the external memory bandwidth requirement, image pyramid data are stored every other scale and a shared data-reuse structure is also introduced for patch loading. Besides, we design a score recorder to refine the keypoints in different regions adaptively. As a result, the relatively balanced distribution of keypoints in a frame is in favor of establishing keypoint correspondences. The proposed architecture is implemented on a Zynq-family FPGA with 100MHz and can extract 1000 features in full-HD images at 42fps. An external memory bandwidth of 1.96Gbps and internal memory of 583.4Kbits are suitable for practical embedded applications.