Inter-frame prediction using motion hints

Abstract

We recently proposed a novel approach that employs motion hints for inter-frame prediction. Motion hints are a loose and global description of motion communicated as metadata; they specify motion but they leave it to the client/decoder to find the exact locations where motion is applicable. This work proposes a multi-scale approach for identifying these exact locations, which are then used with the available reference frames to generate an inter-frame prediction. The proposed approach is localized and robust to noise and illumination changes. The scheme of this work is applicable to close-loop prediction, but it is more useful in open-loop prediction scenarios, such as using prediction in conjunction with remote browsing of surveillance footage, communicated by a JPIP server. We show that, with reasonably accurate motion, it is possible to produce good inter-frame predictions visually and in terms of PSNR.