2018 · 24 citations · 14 references
EngineeringComputer ScienceRobotics EngineeringArtificial IntelligenceHumanoid RoboticsRoboticsGesture RecognitionHuman-robot CommunicationHuman-robot Interaction LearningMultimodal InteractionPerception For Humanoid RobotsSpeech InteractionGesture Recognition In Human-robot InteractionMicrophonesMultimodal Human-robot InteractionRobot Multimodal InteractionRobustnessSensory ModalitiesSocially Interactive RobotsSpeech-based Human-robot Interaction
Pepper is a humanoid robot, specifically designed for social interaction, that has been deployed in a variety of public environments. A programmable version of Pepper is also available, enabling our focused research on perception and behavior robustness and capabilities of an interactive social robot. We address Pepper perception by integrating state-of-the-art vision and speech recognition systems and experimentally analyzing their effectiveness. As we recognize limitations of the individual perceptual modalities, we introduce a multi-modality approach to increase the robustness of human social interaction with the robot. We combine vision, gesture, speech, and input from an onboard tablet, a remote mobile phone, and external microphones. Our approach includes the proactive seeking of input from a different modality, adding robustness to the failures of the separate components. We also introduce a learning algorithm to improve communication capabilities over time, updating speech recognition through social interactions. Finally, we realize the rich robot body-sensory data and introduce both a nearest-neighbor and a deep learning approach to enable Pepper to classify and speak up a variety of its own body motions. We view the contributions of our work to be relevant both to Pepper specifically and to other general social robots.
14
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
YOLO9000: Better, Faster, Stronger
Joseph Redmon, Ali Farhadi · 2017 · 19K citations
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Zhe Cao, Tomas Simon, Shih-En Wei et al. · 2017 · 7.3K citations
Introduction to the Theory of Computation
Michael Sipser · ACM SIGACT News · 1996 · 2.8K citations
Mutual disambiguation of recognition errors in a multimodel architecture
Sharon Oviatt · 1999 · 315 citations