Concepedia

Multimodal Speech Interaction

Multimodal Speech Interaction is a systematic framework for human-computer interaction that integrates speech with complementary modalities such as gesture, gaze, or facial expression to enable more natural and robust communication, distinguished from unimodal speech systems by its capacity to process and synchronize multiple streams of perceptual input in interactive contexts.

266

Publications

10.2K

Citations

1.3K

Authors

361

Institutions

Publications per year

2017–2026

81

Parents

Publications

266

Designing the User Interface for Multimodal Speech and Pen-Based Gesture Applications: State-of-the-Art Systems and Future Research Directions

Sharon Oviatt, Philip R. Cohen, Lizhong Wu et al. · Human-Computer Interaction · 2000 · 374 citations

, , +16

Speech gesture generation from the trimodal context of text, audio, and speaker identity

Youngwoo Yoon, Bok Cha, Joo-Haeng Lee et al. · ACM Transactions on Graphics · 2020 · 304 citations · Full text

, , +17

Polysensory Interactions along Lateral Temporal Regions Evoked by Audiovisual Speech

Tricia M. Wright · Cerebral Cortex · 2003 · 284 citations

, , +17

Speech and gesture share the same communication system

Paolo Bernardis, Maurizio Gentilucci · Neuropsychologia · 2005 · 261 citations

, , +12

The Role of Synchrony and Ambiguity in Speech–Gesture Integration during Comprehension

Boukje Habets, Sotaro Kita, Zeshu Shao et al. · Journal of Cognitive Neuroscience · 2010 · 160 citations · Full text

, , +16

1–5 of 266

Authors

1.3K

  1. SO
    Sharon Oviatt

    Oregon Health & Science University, United States

    7 publications · 626 citations

    7

    h-index

  2. JB
    Jonas Beskow

    KTH Royal Institute of Technology, Sweden

    6 publications · 300 citations

    6

    h-index

  3. YY
    Y. Yemez

    Koç University, Türkiye

    4 publications · 156 citations

    4

    h-index

  4. RL
    Rebecca Lunsford

    Oregon Health & Science University, United States

    4 publications · 191 citations

    4

    h-index

  5. EE
    Engin Erzin

    Koç University, Türkiye

    4 publications · 156 citations

    4

    h-index

1–5 of 1.3K

Institutions

361

  1. Carnegie Mellon University

    Pittsburgh, United States

    12 h-index

    31 publications

    1.6K citations

  2. KTH Royal Institute of Technology

    Stockholm, Sweden

    9 h-index

    18 publications

    480 citations

  3. Microsoft (United States)

    Redmond, United States

    8 h-index

    19 publications

    682 citations

  4. Centre National de la Recherche Scientifique

    Paris, France

    7 h-index

    19 publications

    571 citations

  5. The University of Tokyo

    Tokyo, Japan

    6 h-index

    12 publications

    229 citations

  6. Radboud University Nijmegen

    Nijmegen, The Netherlands

    6 h-index

    9 publications

    446 citations

  7. University of Washington

    Seattle, United States

    6 h-index

    10 publications

    343 citations

  8. Oregon Health & Science University

    Portland, United States

    6 h-index

    19 publications

    1.3K citations

1–8 of 361

Venues

  1. 5

    h-index

  2. The Journal of the Acoustical Society of America

    JASA · 4 publications · 119 citations

    4

    h-index

  3. Speech Communication

    3 publications · 99 citations

    3

    h-index

  4. Journal of Cognitive Neuroscience

    3 publications · 258 citations

    3

    h-index

  5. IEEE Transactions on Multimedia

    3 publications · 122 citations

    3

    h-index

1–5