Validity and reliability of an accelerometer-based player tracking device

TLDR

The study sought to assess intra‑ and inter‑device accuracy and reliability of wearable athletic tracking devices in controlled laboratory conditions. Nineteen Catapult OptimEye S5 accelerometers were mounted on a shaker table and subjected to 60 oscillation tests per unit across three orthogonal directions and four acceleration levels, recording data at 100 Hz to compare device‑measured peak accelerations, PlayerLoad™, and calculated player load. Devices showed excellent intradevice reliability but mixed interdevice reliability, with mean differences up to 15 % in PlayerLoad™ and interdevice effect sizes ranging from small to large (ICC 0.77–1.0); these findings highlight the need for industry‑wide standards and regular accuracy assessments.

Abstract

This study aimed to determine the intra- and inter-device accuracy and reliability of wearable athletic tracking devices, under controlled laboratory conditions. A total of nineteen portable accelerometers (Catapult OptimEye S5) were mounted to an aluminum bracket, bolted directly to an Unholtz Dickie 20K electrodynamic shaker table, and subjected to a series of oscillations in each of three orthogonal directions (front-back, side to side, and up-down), at four levels of peak acceleration (0.1g, 0.5g, 1.0g, and 3.0g), each repeated five times resulting in a total of 60 tests per unit, for a total of 1140 records. Data from each accelerometer was recorded at a sampling frequency of 100Hz. Peak accelerations recorded by the devices, Catapult PlayerLoad™, and calculated player load (using Catapult's Cartesian formula) were used for the analysis. The devices demonstrated excellent intradevice reliability and mixed interdevice reliability. Differences were found between devices for mean peak accelerations and PlayerLoad™ for each direction and level of acceleration. Interdevice effect sizes ranged from a mean of 0.54 (95% CI: 0.34–0.74) (small) to 1.20 (95% CI: 1.08–1.30) (large) and ICCs ranged from 0.77 (95% CI: 0.62–0.89) (very large) to 1.0 (95% CI: 0.99–1.0) (nearly perfect) depending upon the magnitude and direction of the applied motion. When compared to the player load determined using the Cartesian formula, the Catapult reported PlayerLoad™ was consistently lower by approximately 15%. These results emphasize the need for industry wide standards in reporting validity, reliability and the magnitude of measurement errors. It is recommended that device reliability and accuracy are periodically quantified.