Designing a vision-based mobile interface for in-store shopping

Abstract

Due to the situated nature of mobile applications, designing them requires more emphasis on users' cognitive load and interaction style. Considering that users can only devote limited and fragmented attention to mobile interface when moving between locations, what interaction-styles and services are appropriate to a specific user scenario? To explore this issue for in-store shopping, we designed a vision-based mobile interface for supporting shopper's communicational and organizational requirements on-the-go. With this interface, the physical objects can be automatically recognized by the camera phone in real time, so that shoppers can easily access related internet services. In this paper, we present an ethnographic study from which the design rationale is generated, and a formative evaluation to understand how mobile visual interface can be used in the field. The issues uncovered and lessons learned are applicable to our design improvement. Moreover, we use this to motivate the discussion on vision-based mobile interfaces in general, including embodied interaction and alternate interfaces.