2003 · 26 citations · 13 references
EngineeringComputer ScienceRobotics EngineeringHuman-robot InteractionRobot ControlRoboticsGesture RecognitionHuman-machine InterfacesHuman-robot CommunicationHuman-robot Interaction DesignMultimodal InteractionRobot ManipulationGesture Recognition In Human-robot InteractionIntention DetectionInteractive Task ExecutionMultimodal Human-robot InteractionMultimodal User InterfacesRobot Multimodal InteractionSensor Data ProcessingSkill-based Robot Programming
This paper introduces a novel approach to program a robot interactively through a multi-modal interface. The key characteristic of this approach is that the user can provide feedback interactively at any time - during both the programming and the execution phase. The framework takes a three-step approach to the problem: multi-modal recognition, intention interpretation, and prioritized task execution. The multi-modal recognition module translates hand gestures and spontaneous speech into a structured symbolic data stream without abstracting away the user's intent. The intention interpretation module selects the appropriate primitives to generate a task based on the user's input, the system's current state, and robot sensor data. Finally, the prioritized task execution module selects and executes skill primitives based on the system's current state, sensor inputs, and prior tasks. The framework is demonstrated by interactively controlling and programming a vacuum-cleaning robot.
13
Taming recognition errors with a multimodal interface
Sharon Oviatt · Communications of the ACM · 2000 · 301 citations · Full text
Understanding purposeful human motion
Christopher R. Wren, Brian Clarkson, Alex Pentland · 2002 · 77 citations
Computing, Computer Science, Physics +15
Manipulation task primitives for composing robot skills
James D. Morrow, P.K. Khosla · 2002 · 70 citations
An architecture for gesture-based control of mobile robots
Soshi Iba, J. Michael Vande Weghe, Christiaan J. J. Paredis et al. · 2003 · 61 citations