2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · 2021 · 21 citations · 5 references
EducationEducational TechnologyPerforming ArtsAssistive TechnologyAugmented Reality In EducationComputer-assisted InstructionAugmented RealityAugmented Reality PerformanceVirtual RealityHand Motion CaptureHead-mounted Display RenderingHidden Markov Model TrainingMidi Data RepresentationMidi Systems
Multimedia instrument training has gained great momentum benefiting from augmented and/or virtual reality (AR/VR) technologies. We present an AR-based individual training system for piano performance that uses only MIDI data as input. Based on fingerings decided by a pre-trained Hidden Markov Model (HMM), the system employs musical prior knowledge to generate natural-looking 3D animation of hand motion automatically. The generated virtual hand demonstrations are rendered in head-mounted displays and registered with a piano roll. Two user studies conducted by us show that the system requires relatively less cognitive load and may increase learning efficiency and quality.
5
Katja Rogers, Amrei Röhlig, Matthias Weing et al. · 2014 · 73 citations
A system for automatic animation of piano performances
Yuanfeng Zhu, Ajay Sundar Ramakrishnan, Bernd Hamann et al. · Computer Animation and Virtual Worlds · 2012 · 28 citations
CG animation for piano performance
Nozomi Kugimoto, Rui Miyazono, Kosuke Omori et al. · 2009 · 17 citations