Knowing the user's every move

TLDR

Detailed, fine‑grained user interaction data is needed beyond classical log files, and tracking must be non‑intrusive and compatible with existing server and browser setups. The study investigates monitoring detailed user interaction with standard web technologies to enable implicit interaction and ease usability evaluation outside the lab. The authors implement a proxy that injects JavaScript into pages to collect mouse movements, keyboard input, and other data, enabling tasks such as proficiency classification and form completion time assessment. A case study demonstrates the usefulness of this detailed tracking approach.

Abstract

In this paper, we investigate how detailed tracking of user interaction can be monitored using standard web technologies. Our motivation is to enable implicit interaction and to ease usability evaluation of web applications outside the lab. To obtain meaningful statements on how users interact with a web application, the collected information needs to be more detailed and fine-grained than that provided by classical log files. We focus on tasks such as classifying the user with regard to computer usage proficiency or making a detailed assessment of how long it took users to fill in fields of a form. Additionally, it is important in the context of our work that usage tracking should not alter the user's experience and that it should work with existing server and browser setups. We present an implementation for detailed tracking of user actions on web pages. An HTTP proxy modifies HTML pages by adding JavaScript code before delivering them to the client. This JavaScript tracking code collects data about mouse movements, keyboard input and more. We demonstrate the usefulness of our approach in a case study.