Detecting Behavior Anomalies in Graphical User Interfaces

Abstract

When interacting with user interfaces, do users always get what they expect? For each user interface element in thousands of Android apps, we extracted the Android APIs they invoke as well as the text shown on their screen. This association allows us to detect outliers: User interface elements whose text, context or icon suggests one action, but which actually are tied to other actions. In our evaluation of tens of thousands of UI elements, our BACKSTAGE prototype discovered misleading random UI elements with an accuracy of 73%.