Configurable Image Rectification and Disparity Refinement for Stereo Vision

Abstract

Stereo matching is an essential topic in computer vision. However, the high complexity and massive computation restrict its applicability in real-time. This brief presents a pixel-level pipeline coprocessor for configurable binocular stereo rectification, region-optimized semi-global matching (SGM), left-right consistency checking (LRC), and disparity refinement. The contributions are summarized as follows: 1) Instead of storing the entire image of a large number of lines, the buffer size can be significantly reduced by the adjustment range computed according to the pre-calibrated coefficients. 2) An approximate divider with Newton iterations can reach the precision of 10−7 for high accuracy rectification with low latency. 3) A four-layer parallel two-stage pipeline is leveraged to address the computational bottleneck of the SGM, i.e., cost aggregation. 4) The pseudo-dual-port SRAMs and stacks mechanism solve the problem of inverse pixel flow in hole filling with low resource consumption. Finally, the proposed architecture is implemented on an Intel Stratix-IV FPGA and can process 144 frames ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1024\mathbf {\times }768$ </tex-math></inline-formula> -pixel) per second at a maximum working frequency of 113.13 MHz.