Characterizing Information Seeking Processes with Multiple Physiological Signals

Abstract

Information access systems are getting complex, and our understanding of user\nbehavior during information seeking processes is mainly drawn from qualitative\nmethods, such as observational studies or surveys. Leveraging the advances in\nsensing technologies, our study aims to characterize user behaviors with\nphysiological signals, particularly in relation to cognitive load, affective\narousal, and valence. We conduct a controlled lab study with 26 participants,\nand collect data including Electrodermal Activities, Photoplethysmogram,\nElectroencephalogram, and Pupillary Responses. This study examines\ninformational search with four stages: the realization of Information Need\n(IN), Query Formulation (QF), Query Submission (QS), and Relevance Judgment\n(RJ). We also include different interaction modalities to represent modern\nsystems, e.g., QS by text-typing or verbalizing, and RJ with text or audio\ninformation. We analyze the physiological signals across these stages and\nreport outcomes of pairwise non-parametric repeated-measure statistical tests.\nThe results show that participants experience significantly higher cognitive\nloads at IN with a subtle increase in alertness, while QF requires higher\nattention. QS involves demanding cognitive loads than QF. Affective responses\nare more pronounced at RJ than QS or IN, suggesting greater interest and\nengagement as knowledge gaps are resolved. To the best of our knowledge, this\nis the first study that explores user behaviors in a search process employing a\nmore nuanced quantitative analysis of physiological signals. Our findings offer\nvaluable insights into user behavior and emotional responses in information\nseeking processes. We believe our proposed methodology can inform the\ncharacterization of more complex processes, such as conversational information\nseeking.\n