Towards multi-modal anticipatory monitoring of depressive states through the analysis of human-smartphone interaction

TLDR

Smartphone advances now allow passive, real‑time monitoring of user behavior at unprecedented granularity, and researchers have begun using location, call, SMS, and app‑usage logs to infer depressive states. This study proposes a multi‑modal smartphone‑based monitoring framework for depressive states and outlines the challenges of predicting depression through such sensing. The framework combines multiple sensing modalities—location, call, SMS, and application usage—drawing on a literature review of prior depression‑monitoring studies. Initial results from an ongoing study indicate that depressive states are associated with specific smartphone interaction patterns.

Abstract

Remarkable advances in smartphone technology, especially in terms of passive sensing, have enabled researchers to passively monitor user behavior in real-time and at a granularity that was not possible just a few years ago. Recently, different approaches have been proposed to investigate the use of different sensing and phone interaction features, including location, call, SMS and overall application usage logs, to infer the depressive state of users. In this paper, we propose an approach for monitoring of depressive states using multi-modal sensing via smartphones. Through a brief literature review we show the sensing modalities that have been exploited in the past studies for monitoring depression. We then present the initial results of an ongoing study to demonstrate the association of depressive states with the smartphone interaction features. Finally, we discuss the challenges in predicting depression through multimodal mobile sensing.