Exploiting Spatial Redundancy of Image Sensor for Motion Robust rPPG

TLDR

Remote photoplethysmography measures heart rate from skin color changes captured by an RGB camera, but current methods are highly sensitive to motion‑induced color distortions. This study introduces a framework to enhance the motion robustness of rPPG. By treating each pixel as an independent pulse sensor, the method exploits the image sensor’s spatial redundancy through motion‑compensated pixel‑to‑pixel extraction, spatial pruning, and temporal filtering, and is evaluated on 36 diverse benchmark videos. The approach raises SNR from 3.34 to 6.76 dB and increases agreement with reference pulse rate from 55 % to 80 %, a statistically significant improvement that brings performance close to contact sensors while enabling real‑time processing.

Abstract

Remote photoplethysmography (rPPG) techniques can measure cardiac activity by detecting pulse-induced color variations on human skin using an RGB camera. State-of-the-art rPPG methods are sensitive to subject body motions (e.g., motion-induced color distortions). This study proposes a novel framework to improve the motion robustness of rPPG. The basic idea of this paper originates from the observation that a camera can simultaneously sample multiple skin regions in parallel, and each of them can be treated as an independent sensor for pulse measurement. The spatial redundancy of an image sensor can thus be exploited to distinguish the pulse signal from motion-induced noise. To this end, the pixel-based rPPG sensors are constructed to estimate a robust pulse signal using motion-compensated pixel-to-pixel pulse extraction, spatial pruning, and temporal filtering. The evaluation of this strategy is not based on a full clinical trial, but on 36 challenging benchmark videos consisting of subjects that differ in gender, skin types, and performed motion categories. Experimental results show that the proposed method improves the SNR of the state-of-the-art rPPG technique from 3.34 to 6.76 dB, and the agreement ( ±1.96σ) with instantaneous reference pulse rate from 55% to 80% correct. ANOVA with post hoc comparison shows that the improvement on motion robustness is significant. The rPPG method developed in this study has a performance that is very close to that of the contact-based sensor under realistic situations, while its computational efficiency allows real-time processing on an off-the-shelf computer.