ComputingEducational TechnologyInstructionComputer-assisted InstructionInteraction DesignNatural Language ProcessingMobile Application DevelopmentMobile Human-computer InteractionMobile User ExperienceMultimodal InteractionInfrastructure SimulationVisual Instructions
We present HelpViz, a tool for generating contextual visual mobile tutorials from text-based instructions that are abundant on the web. HelpViz transforms text instructions to graphical tutorials in batch, by extracting a sequence of actions from each text instruction through an instruction parsing model, and executing the extracted actions on a simulation infrastructure that manages an array of Android emulators. The automatic execution of each instruction produces a set of graphical and structural assets, including images, videos, and metadata such as clicked elements for each step. HelpViz then synthesizes a tutorial by combining parsed text instructions with the generated assets, and contextualizes the tutorial to user interaction by tracking the user’s progress and highlighting the next step. Our experiments with HelpViz indicate that our pipeline improved tutorial execution robustness and that participants preferred tutorials generated by HelpViz over text-based instructions. HelpViz promises a cost-effective approach for generating contextual visual tutorials for mobile interaction at scale.
36
THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.<sup>1</sup>
Paul Jaccard · New Phytologist · 1912 · 4.5K citations · Full text
Life Sciences, Botany, Ecology +10
Jerry Alan Fails, Dan R. Olsen · 2003 · 604 citations
Harry N. Boone, Deborah Boone · Journal of Extension · 2012 · 576 citations · Full text
Education, Educational Assessment, Educational Measurement +16
Biplab Deka, Zifeng Huang, Chad Franzen et al. · 2017 · 461 citations · Full text
Olivier Bau, Wendy E. Mackay · 2008 · 261 citations
Arts, Design, Educational Technology +17