Metrics for text entry research

TLDR

The paper identifies limitations of the minimum string distance error rate and keystrokes per character, and proposes a new error‑analysis framework to address them. The framework analyzes presented text, keystrokes, and transcribed output, introducing a unified total error rate (corrected and not corrected) alongside measures of correction efficiency, participant conscientiousness, and bandwidth usage. A text‑entry study demonstrates the applicability of the new framework, showing its ability to capture error dynamics more comprehensively than previous metrics.

Abstract

We describe and identify shortcomings in two statistics recently introduced to measure accuracy in text entry evaluations: the minimum string distance (MSD) error rate and keystrokes per character (KSPC). To overcome the weaknesses, a new framework for error analysis is developed and demonstrated. It combines the analysis of the presented text, input stream (keystrokes), and transcribed text. New statistics include a unified total error rate, combining two constituent error rates: the corrected error rate (errors committed but corrected) and the not corrected error rate (errors left in the transcribed text). The framework includes other measures including error correction efficiency, participant conscientiousness, utilised bandwidth, and wasted bandwidth. A text entry study demonstrating the new methodology is described.