MotionHacker: Motion sensor based eavesdropping on handwriting via smartwatch

Abstract

Motion sensors may impose the danger of hand-writing leakage if the smartwatch installs a malicious app. By presenting a proof-of-concept system: MotionHacker, we show the usage of smartwatch app to record the motions and extract handwriting-specific features for machine learning based analysis. MotionHacker is targeted on user-independent handwriting recognition and word-level estimation for the content of the victim's handwriting, where the only limit is that victim's handwriting follows print style. Furthermore, the targeted handwriting is lowercase letters, which are more difficult than capital letters. Our experimental results show that the average accuracy of word recognition is 32.8% for 5 victims writing two graphs from a novel and a research paper each. When counting the word prediction results of top-5 items, the word recognition accuracy reaches 48.8%. The results confirm the danger of handwriting content leakage from smartwatches' motion sensors.