Acoustic tracking of hand activities on surfaces

Abstract

Many common forms of activities are tactile in their nature. We touch, grasp, and interact with a plethora of objects every day. Some of those objects are registering our activities, such as the millions of touch screens we are using every day. Adding perception to arbitrary objects is an active area of research, with a variety of technologies in use. Acoustic sensors, such as microphones, react to mechanical waves propagating through a medium. By attaching an acoustic sensor to a surface, we can analyze activities on this medium. In this paper, we present signal analysis and machine learning methods that enable us to detect a variety of interaction events on a surface. We extend from previous work, by combining swipe and touch detection in a single method, for the latter achieving an accuracy between 91% and 99% with a single microphone and 97% to 100% with two microphones.