Exploring Contextual Engagement for Trauma Recovery

TLDR

Engagement estimation from facial data has been studied across domains, and psychological research shows it is a key predictor of treatment effectiveness and behavioral outcomes. This study investigates whether face‑based engagement models are context‑free or task‑specific by analyzing user engagement during relaxation and trigger modules of a trauma‑recovery program. Using over 8 million video frames from 110 subjects and 800 self‑reports, the authors train LSTM sequence models on facial Action Units, extended with a brief Profile of Mood States to predict engagement. The results show that engagement prediction is highly contextual, with models trained on one task performing poorly on another, and that context‑specific models achieve higher accuracy in web‑based trauma‑recovery tools.

Abstract

A wide range of research has used face data to estimate a person's engagement, in applications from advertising to student learning. An interesting and important question not addressed in prior work is if face-based models of engagement are generalizable and context-free, or do engagement models depend on context and task. This research shows that context-sensitive face-based engagement models are more accurate, at least in the space of web-based tools for trauma recovery. Estimating engagement is important as various psychological studies indicate that engagement is a key component to measure the effectiveness of treatment and can be predictive of behavioral outcomes in many applications. In this paper, we analyze user engagement in a trauma-recovery regime during two separate modules/tasks: relaxation and triggers. The dataset comprises of 8M+ frames from multiple videos collected from 110 subjects, with engagement data coming from 800+ subject self-reports. We build an engagement prediction model as sequence learning from facial Action Units (AUs) using Long Short Term Memory (LSTMs). Our experiments demonstrate that engagement prediction is contextual and depends significantly on the allocated task. Models trained to predict engagement on one task are only weak predictors for another and are much less accurate than context-specific models. Further, we show the interplay of subject mood and engagement using a very short version of Profile of Mood States (POMS) to extend our LSTM model.