ComputingComputer ScienceEducational TechnologyComputer VisionNatural Language ProcessingVisual AttentionMultimodal InteractionTopic ModelingVideo AnalysisLecture Video AnalysisMultimodal Topic ModelingSaliency DetectionSaliency EstimationUser Study EvaluationVideo SegmentationVisual Saliency
In this paper we propose a multimodal method called MMToC for automatically creating a table of content for educational videos. MMToC defines and quantifies word saliency for visual words extracted from the slides and spoken words obtained from the speech transcript. The saliency scores from these two modalities are combined to obtain a ranked list of salient words. These ranked words along with their saliency scores are used to formulate a topic segmentation cost function. The cost function is optimized using a dynamic program framework to obtain the topic segments of the video. These segments are labelled with their corresponding topic names for creating the table of content. We perform experiments on 24 hours of lectures spread across 23 videos ranging over 20-75 minutes duration each. We compare the proposed method with LDA-based video segmentation approaches and show that the proposed MMToC method is significantly better (F-score improvement of 0.19 and 0.24 on two datasets). We also perform a user study to demonstrate the effectiveness of MMToC for navigating educational videos.
31
An algorithm for suffix stripping
Martin Porter · Program electronic library and information systems · 1980 · 8.1K citations
David M. Blei · Communications of the ACM · 2012 · 5.5K citations
Feature-rich part-of-speech tagging with a cyclic dependency network
Kristina Toutanova, Dan Klein, Christopher D. Manning et al. · 2003 · 2.9K citations · Full text
Quary Expansion Using Local and Global Document Analysis
Jinxi Xu, W. Bruce Croft · ACM SIGIR Forum · 2017 · 1.3K citations