WiGest: A ubiquitous WiFi-based gesture recognition system

TLDR

WiGest builds on the emerging field of WiFi‑based gesture recognition, distinguishing itself by using only standard, unmodified WiFi hardware without requiring training data. The study introduces WiGest, a system that detects in‑air hand gestures by monitoring changes in WiFi signal strength around a mobile device. WiGest extracts signal‑change primitives from WiFi RSSI, constructs independent gesture families, maps them to application actions, and mitigates noise, interference, and polarity shifts through signal cleaning and adaptive algorithms, as demonstrated in a laptop‑based prototype tested in office and apartment settings. The prototype achieves 87.5 % primitive detection accuracy with a single access point, rising to 96 % with three APs, and attains 96 % gesture classification accuracy on a multimedia player, remaining robust in the presence of other humans.

Abstract

We present WiGest: a system that leverages changes in WiFi signal strength to sense in-air hand gestures around the user's mobile device. Compared to related work, WiGest is unique in using standard WiFi equipment, with no modifications, and no training for gesture recognition. The system identifies different signal change primitives, from which we construct mutually independent gesture families. These families can be mapped to distinguishable application actions. We address various challenges including cleaning the noisy signals, gesture type and attributes detection, reducing false positives due to interfering humans, and adapting to changing signal polarity. We implement a proof-of-concept prototype using off-the-shelf laptops and extensively evaluate the system in both an office environment and a typical apartment with standard WiFi access points. Our results show that WiGest detects the basic primitives with an accuracy of 87.5% using a single AP only, including through-the-wall non-line-of-sight scenarios. This accuracy increases to 96% using three overheard APs. In addition, when evaluating the system using a multi-media player application, we achieve a classification accuracy of 96%. This accuracy is robust to the presence of other interfering humans, highlighting WiGest's ability to enable future ubiquitous hands-free gesture-based interaction with mobile devices.