Programming by voice

Abstract

This paper introduces a voice-driven tool applied to an Initial Programming Environment (IPE), which gives motorically challenged individuals the opportunity to learn programming skills; in particular, our project allows programming by voice within Scratch. Although the native Scratch environment allows users to create a program by arranging graphical blocks logically, such visual languages are completely dependent on the use of a mouse and keyboard. This modality of interaction limits users based on physical abilities. Our solution is a tool, called Myna, which is a voice-driven Java application executed parallel to Scratch. Myna processes voice commands from the user, interprets those commands according to a pre-defined grammar, and simulates synonymous actions of a mouse and keyboard within Scratch. The resulting environment assists those with a motor disability (particularly young children) in learning the joy of programming. This extended abstract describes the motivation behind the project, a technical description of Myna, and defines the current work in progress.