Phase-based video motion processing

Neal Wadhwa, Michael Rubinstein, Frédo Durand, William T. Freeman

ACM Transactions on Graphics · 2013 · 798 citations · 16 references

DOIFull text

Open access

Concepts

TL;DR

The authors present a method for manipulating subtle video motions by analyzing motion in complex‑valued image pyramids. The technique tracks phase variations in a complex‑valued steerable pyramid, temporally processes them to amplify or attenuate motion, and avoids optical flow while allowing larger amplification and lower noise sensitivity compared to Eulerian Video Magnification. The method enables broader applications, as shown on synthetic and natural videos, and offers benefits for scientific analysis, visualization, and video enhancement.

Abstract

We introduce a technique to manipulate small movements in videos based on an analysis of motion in complex-valued image pyramids. Phase variations of the coefficients of a complex-valued steerable pyramid over time correspond to motion, and can be temporally processed and amplified to reveal imperceptible motions, or attenuated to remove distracting changes. This processing does not involve the computation of optical flow, and in comparison to the previous Eulerian Video Magnification method it supports larger amplification factors and is significantly less sensitive to noise. These improved capabilities broaden the set of applications for motion processing in videos. We demonstrate the advantages of this approach on synthetic and natural video sequences, and explore applications in scientific analysis, visualization and video enhancement.

References

16