2021 · 27 citations · 38 references
EducationEducational TechnologyInstructionComputer-assisted InstructionHuman-computer InteractionInstructional DesignConversational User InterfacesInteractive Learning EnvironmentsMultimedia AuthoringMultimedia Instructional ContentMultimedia LearningVideo-based InstructionInteractive Multimedia CoursewareInteractive Video AuthoringInteractive Video LearningUser-generated Video ContentVideo Editing For InstructionVideo TutorialsVisual Instructions
We introduce HowToCut, an automatic approach that converts a Markdown-formatted tutorial into an interactive video that presents the visual instructions with a synthesized voiceover for narration. HowToCut extracts instructional content from a multimedia document that describes a step-by-step procedure. Our method selects and converts text instructions to a voiceover. It makes automatic editing decisions to align the narration with edited visual assets, including step images, videos, and text overlays. We derive our video editing strategies from an analysis of 125 web tutorials and apply Computer Vision techniques to the assets. To enable viewers to interactively navigate the tutorial, HowToCut’s conversational UI presents instructions in multiple formats upon user commands. We evaluated our automatically-generated video tutorials through user studies (N=20) and validated the video quality via an online survey (N=93). The evaluation shows that our method was able to effectively create informative and useful instructional videos from a web tutorial document for both reviewing and following.
38
A deep learning approach for generalized speech animation
Sarah Taylor, Tae‐Hwan Kim, Yisong Yue et al. · ACM Transactions on Graphics · 2017 · 309 citations · Full text
Direct Speech-to-Speech Translation with a Sequence-to-Sequence Model
Jia Ye, Ron J. Weiss, Fadi Biadsy et al. · 2019 · 174 citations
Computing, Computer Science, Corpus-based Translation Studies +13
Designing effective step-by-step assembly instructions
Maneesh Agrawala, Doantam Phan, Julie Heiser et al. · ACM Transactions on Graphics · 2003 · 164 citations
Architecture, Design, Instruction +17
Crowdsourcing step-by-step information extraction to enhance existing how-to videos
Juho Kim, Phu Tran Nguyen, Sarah Weir et al. · 2014 · 128 citations · Full text
Computational video editing for dialogue-driven scenes
Mackenzie Leake, Abe Davis, Anh Minh Truong et al. · ACM Transactions on Graphics · 2017 · 122 citations