Investigating Engagement with In-Video Quiz Questions in a Programming Course

Abstract

In-video quizzes are common in many distance learning platforms, including those from Coursera and EdX. However, the effectiveness of in-video quizzes has not previously been assessed. In this paper, we describe the construction and instrumentation of an Interactive Video Lecture Platform to measure student engagement with in-video quizzes. We also investigate the use of in-video quizzes as an approach to mitigate the lack of feedback available to students and lecturers in videos and traditional lectures. Finally, we evaluate the effectiveness of augmenting video with the ability to answer and receive feedback to quiz questions embedded directly within the video. We observed that student engagement with in-video questions was consistently high (71-86 percent) across two cohorts ( <inline-formula> <tex-math notation="LaTeX">$N_1$</tex-math></inline-formula> = 81, <inline-formula><tex-math notation="LaTeX">$N_2$</tex-math></inline-formula> = 84) with a rate of one question per 8.7 minutes of video. We identified three broad levels of engagement with the quiz questions and four motivations, including challenge seeking and completionism, which explain some of the observed behaviour. The results from this investigation demonstrate that in-video quizzes were successful in creating an engaging and interactive mode of content delivery. We recommend that in-video quizzes be used to increase the interactivity of video content as well as supporting formative assessment within a flipped classroom environment.