Beyond the text: construction and analysis of multi-modal linguistic corpora

Abstract

This paper addresses some of the linguistic and technological procedures and requirements of the next generation of tools for the analysis of spoken linguistic corpora. It reports on preliminary developments of an ESRC funded interdisciplinary project at the University of Nottingham. It specifically focuses on key methodological and technical issues related to the mark-up, coding and representation of multi-modal communication data. The paper builds on traditional approaches in the area of spoken corpus linguistics, which use multi-million word databases of textual renderings of naturally occurring conversation with the aim of identifying recurring patterns of lexical and grammatical patterns within sequences of interactions. The identification, classification and representation of accompanying gestural elements are explored in relation to the expression of active listenership.