The KidsRoom: A Perceptually-Based Interactive and Immersive Story Environment

TLDR

The KidsRoom is a perceptually‑based, interactive narrative playspace for children. The paper aims to describe the KidsRoom system, its underlying technology, and the challenges encountered during development. The system uses multimodal sensory cues—images, music, narration, light, and sound effects—alongside computer‑vision algorithms to detect activity in the room and drive a reactive adventure story, enabling multiple users to interact in a fully automated, non‑encumbering environment. The KidsRoom represents the first multi‑person, fully automated, interactive narrative environment built with non‑encumbering sensors. A demonstration of the project, including videos, images, and sounds from each part of the story, is available online.

Abstract

The KidsRoom is a perceptually-based, interactive, narrative playspace for children. Images, music, narration, light, and sound effects are used to transform a normal child's bedroom into a fantasy land where children are guided through a reactive adventure story. The fully automated system was designed with the following goals: (1) to keep the focus of user action and interaction in the physical and not virtual space; (2) to permit multiple, collaborating people to simultaneously engage in an interactive experience combining both real and virtual objects; (3) to use computer-vision algorithms to identify activity in the space without requiring the participants to wear any special clothing or devices; (4) to use narrative to constrain the perceptual recognition, and to use perceptual recognition to allow participants to drive the narrative; and (5) to create a truly immersive and interactive room environment. We believe the KidsRoom is the first multi-person, fully-automated, interactive, narrative environment ever constructed using non-encumbering sensors. This paper describes the KidsRoom, the technology that makes it work, and the issues that were raised during the system's development. 1 1 A demonstration of the project, which complements the material presented here and includes videos, images, and sounds from each part of the story is available at .