Tracking people with twists and exponential maps

TLDR

The study introduces a visual motion estimation method using product‑of‑exponential‑maps and twist motions to recover high‑degree‑of‑freedom articulated human body configurations from complex video sequences. The method applies product‑of‑exponential‑maps and twist motions within a differential motion estimation framework, validated on image sequences of full‑body movements and visualized by re‑animating a 3D human model. The system solves simple linear systems to robustly recover kinematic degrees of freedom even in noisy, self‑occluded scenes, successfully re‑animating both contemporary full‑body movements and historical Muybridge motion studies, marking the first computer‑vision system to achieve such high‑accuracy recovery on challenging footage.

Abstract

This paper demonstrates a new visual motion estimation technique that is able to recover high degree-of-freedom articulated human body configurations in complex video sequences. We introduce the use of a novel mathematical technique, the product of exponential maps and twist motions, and its integration into a differential motion estimation. This results in solving simple linear systems, and enables us to recover robustly the kinematic degrees-of-freedom in noise and complex self occluded configurations. We demonstrate this on several image sequences of people doing articulated full body movements, and visualize the results in re-animating an artificial 3D human model. We are also able to recover and re-animate the famous movements of Eadweard Muybridge's motion studies from the last century. To the best of our knowledge, this is the first computer vision based system that is able to process such challenging footage and recover complex motions with such high accuracy.