The development and evaluation of a survey to measure user engagement

TLDR

Engaging user experiences are essential for designing interactive systems. The study aims to develop and validate a multidimensional user engagement scale through large‑scale studies in online shopping. The authors constructed a multidimensional engagement scale and evaluated it using reliability analysis, exploratory factor analysis, and structural equation modeling across two large studies (N = 440 and N = 802) in online shopping. The validated scale reveals six interrelated engagement attributes, with perceived usability mediating relationships among them, and can assess software application engagement.

Abstract

Abstract Facilitating engaging user experiences is essential in the design of interactive systems. To accomplish this, it is necessary to understand the composition of this construct and how to evaluate it. Building on previous work that posited a theory of engagement and identified a core set of attributes that operationalized this construct, we constructed and evaluated a multidimensional scale to measure user engagement. In this paper we describe the development of the scale, as well as two large‐scale studies (N=440 and N=802) that were undertaken to assess its reliability and validity in online shopping environments. In the first we used Reliability Analysis and Exploratory Factor Analysis to identify six attributes of engagement: Perceived Usability, Aesthetics, Focused Attention, Felt Involvement, Novelty, and Endurability. In the second we tested the validity of and relationships among those attributes using Structural Equation Modeling. The result of this research is a multidimensional scale that may be used to test the engagement of software applications. In addition, findings indicate that attributes of engagement are highly intertwined, a complex interplay of user‐system interaction variables. Notably, Perceived Usability played a mediating role in the relationship between Endurability and Novelty, Aesthetics, Felt Involvement, and Focused Attention.