The State of Speech in HCI: Trends, Themes and Challenges

TLDR

Speech interfaces are growing in popularity. The study aims to map the trends, themes, findings, and methods of empirical research on speech interfaces in HCI. It achieves this by reviewing 68 research papers. The review shows most studies focus on usability or broader system experiences using self‑report measures, identifies nine core research topics, and highlights gaps such as the need for theoretical frameworks, larger critical mass, design work, multi‑user studies, and improved measurement reliability and real‑world deployment.

Abstract

Speech interfaces are growing in popularity. Through a review of 68 research papers this work maps the trends, themes, findings and methods of empirical research on speech interfaces in HCI. We find that most studies are usability/theory-focused or explore wider system experiences, evaluating Wizard of Oz, prototypes, or developed systems by using self-report questionnaires to measure concepts like usability and user attitudes. A thematic analysis of the research found that speech HCI work focuses on nine key topics: system speech production, modality comparison, user speech production, assistive technology \& accessibility, design insight, experiences with interactive voice response (IVR) systems, using speech technology for development, people's experiences with intelligent personal assistants (IPAs) and how user memory affects speech interface interaction. From these insights we identify gaps and challenges in speech research, notably the need to develop theories of speech interface interaction, grow critical mass in this domain, increase design work, and expand research from single to multiple user interaction contexts so as to reflect current use contexts. We also highlight the need to improve measure reliability, validity and consistency, in the wild deployment and reduce barriers to building fully functional speech interfaces for research.