When fitness trackers don't 'fit'

TLDR

Personal tracking technologies enable users to monitor physical activity, yet many are uncertain about the accuracy of the data they record. The study aims to describe the methods users employ to assess device accuracy, identify seven related problems, and propose design recommendations to improve accuracy assessment for individual users. We analyzed 600 product reviews and conducted 24 user interviews to examine the methods users use to evaluate device accuracy and the problems they encounter. Users’ differing expectations, physical traits, activity types, and lifestyles lead to varied perceptions of accuracy, and the lack of sound mental models causes them to design faulty tests and draw incorrect conclusions.

Abstract

Personal tracking technologies allow users to monitor and reflect on their physical activities and fitness. However, users are uncertain about how accurately their devices track their data. In order to better understand this challenge, we analyzed 600 product reviews and conducted 24 interviews with tracking device users. In this paper, we describe what methods users used to assess accuracy of their tracking devices and identify seven problems they encountered. We found that differences in users' expectations, physical characteristics, types of activities and lifestyle led them to have different perceptions of the accuracy of their devices. With the absence of sound mental models and unclear understanding of the concepts of accuracy and experimental controls, users designed faulty tests and came to incorrect conclusions. We propose design recommendations to better support end-users' efforts to assess and improve the accuracy of their tracking devices as required to suit their individual characteristics and purposes.