Bridging physical and virtual worlds with tagged documents, objects and locations

Abstract

A compelling and provocative vision of the future was presented in Pierre Wellner's video and article on the Digital Desk [8, 9]. Physical office tools such as pens, erasers, books, and paper were seamlessly integrated (or at least almost seamlessly!) with computational augmentation and virtual tools, using projection and image processing. His work, and now our most recent efforts (reported in this paper and [1, 3, 5]), are directed at more seamlessly bridging the gulf between physical and virtual worlds; an area which we believe represents a key future path for the design of user interfaces. A goal of this work is to seamlessly blend the affordances and strengths of physically manipulatable objects with virtual environments or artifacts, thereby leveraging the particular strengths of each.The goal of this video is to visually present scenarios of a number of working physical prototypes we have designed and built which computationally augment everyday objects to support casual interaction using natural manipulations. Unlike previous work [2, 4, 8, 9], we have tried to build invisible interfaces that have little reliance on specialized single-user environments and/or display projection, or custom-designed objects. To this end, we start with everyday objects and embed computation in them in the ubiquitous computing tradition founded at PARC [6, 7]. We have combined four technologies (RFID identifier tags and readers, RF networking, infrared beacons, and portable computing) in a seamless and tightly integrated way. This combination has not been discussed in the literature and is only now being experimented with in research labs working on user interface design.