Low-complexity macroblock mode selection for H.264/AVC encoders

Abstract

Lagrangian minimization is one of the most powerful tools for rate-distortion optimal coding mode selection. However, in the latest video compression standard H.264/AVC, the total calculation for costs of all candidate modes may become huge since the new standard supports a large number of coding modes. In this paper, we propose an algorithm for fast coding mode selection in H.264/AVC encoders by reducing the number of candidate modes. We also present a rate estimation method for further reduction of Lagrangian cost calculation. Extensive experiments show that the proposed methods can reduce the execution time of mode selection in H.264/AVC reference software by 85% on the average while average PSNR loss is only 0.07 dB.