The Table is The Score: An Augmented-Reality Interface for Real-Time, Tangible, Spectrographic Performance

Abstract

Real-time performance instruments for creating and sonifying spectrographic images have generally taken the form of stylus-based drawing interfaces, or camera-based systems which treat a live video image as a spectrogram. Drawing-based approaches afford great precision in specifying the temporal and pitch structures of spectral events, but can be cumbersome, as they only accept input from a single point; camera-based approaches offer quick flexibility in all-around image improvisation, but poor compositional precision because of inadequate visual feedback to the user. In this paper, I present a camera-based spectrographic performance instrument which affords both compositional precision and improvisatory flexibility. This is made possible through an augmented reality (AR) projection overlaid onto and carefully aligned with a dry-erase performance surface.