2024 · 31 citations · 27 references
EngineeringRobotics EngineeringSpeech-language PathologyAssistive TechnologyHuman-robot InteractionRoboticsAssistive RoboticsConversational User InterfacesHuman-machine InterfacesHuman-robot Interaction DesignMultimodal InteractionRehabilitation RoboticsAssistive Robot DesignAssistive RobotsCommandsLarge Language ModelsRobot Code GenerationSpeech-based Human-robot InteractionSpeech-based InteractionTask Planning
Physically assistive robots present an opportunity to significantly increase the well-being and independence of individuals with motor impairments or other forms of disability who are unable to complete activities of daily living. Speech interfaces, especially ones that utilize Large Language Models (LLMs), can enable individuals to effectively and naturally communicate high-level commands and nuanced preferences to robots. Frameworks for integrating LLMs as interfaces to robots for high level task planning and code generation have been proposed, but fail to incorporate human-centric considerations which are essential while developing assistive interfaces. In this work, we present a framework for incorporating LLMs as speech interfaces for physically assistive robots, constructed iteratively with 3 stages of testing involving a feeding robot, culminating in an evaluation with 11 older adults at an independent living facility. We use both quantitative and qualitative data from the final study to validate our framework and additionally provide design guidelines for using LLMs as speech interfaces for assistive robots. Videos, code, and supporting files are located on our project website1
27
Code as Policies: Language Model Programs for Embodied Control
Jacky Liang, Wenlong Huang, Fei Xia et al. · 2023 · 569 citations
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Ishika Singh, Valts Blukis, Arsalan Mousavian et al. · 2023 · 511 citations
Luca Gualtieri, Erwin Rauch, Renato Vidoni · Robotics and Computer-Integrated Manufacturing · 2020 · 409 citations · Full text
Engineering, Industrial Engineering, Manufacturing Engineering +17
TidyBot: personalized robot assistance with large language models
Jimmy Wu, Rika Antonova, Adam Kan et al. · Autonomous Robots · 2023 · 190 citations