Natural-Language-Instructed Industrial Task Execution

Abstract

To effectively cooperate with a human, advanced manufacturing machines are expected to execute the industrial tasks according to human natural language (NL) instructions. However, NL instructions are not explicit enough to be understood and are not complete enough to be executed, leading to incorrected executions or even execution failure. To address these problems for better execution performance, we developed a Natural-Language-Instructed Task Execution (NL-Exe) method. In NL-Exe, semantic analysis is adopted to extract task-related knowledge, based on what human NL instructions are accurately understood. In addition, logic modeling is conducted to search the missing execution-related specifications, with which incomplete human instructions are repaired. By orally instructing a humanoid robot Baxter to perform industrial tasks “drill a hole” and “clean a spot”, we proved that NL-Exe could enable an advanced manufacturing machine to accurately understand human instructions, improving machine’s performance in industrial task execution.