Acting on a visual world : the role of perception in multimodal HCI

Frédéric Wolff, Antonella De Angeli, Laurent Romary

1998 · 13 citations · 7 references

Full text

Open access

Concepts

Abstract

Following the ecological approach to visual perception, this paper investigates multimodal referring acts in Human-Computer Interaction. Preliminary results from a simulation experiment allow to: (a) clarify the effect perceptual organization on multimodal communication; (b) provide guidelines for designing effective multimodal interfaces. Demonstrating that both the verbal and the gestural part of a referential act are influenced by perception, we confirm the need and the utility of taking into account perceptual organization to analyze referential expressions in a more robust way. As a conclusion, we show how these results can lead to an actual implementation of a gesture interpretation module directed by the reference analysis process.

References

7