JavaSketchIt: Issues in Sketching the Look of User Interfaces

Abstract

We present a visual approach to layout static components of user interfaces as hand-drawn compositions of simple geo-metric shapes, based on sketch recognition. We have de-fined a visual grammar using drawing data from target users, where we tried to figure out how people sketch interfaces and what combinations of shapes are more commonly used to de-fine widgets. From these we built our grammar and imple-mented a prototype, JavaSketchIt, that allows creating user interfaces through hand-drawn geometric shapes, identified by a gesture recognizer. This prototype generates a Java in-terface, whose layout can be beautified using an a posteri-ori set of grammar rules (e.g. to align and group objects). To validate our approach, we conducted usability studies to compare our approach with a commercial system (JBuilder). Besides a measurable speed advantage in drawing interfaces, users found our system more comfortable, natural and intu-itive to use, than the competing product, as demonstrated by post–experiment inquiries.