Scientific Data · 2023 · 53 citations · 18 references
ComputingComputer ScienceMedical EducationClinical InformaticsComputer VisionNatural Language ProcessingAnnotationMulti-task LearningMultimodal InteractionQuestion AnsweringVideo AnalysisVisual Question AnsweringMedical Visual Question AnsweringMultimodal ClassificationMultimodal FusionMultimodal Machine LearningMultimodal Question AnsweringVideo ClassificationVideo Question Answering
This paper introduces a new challenge and datasets to foster research toward designing systems that can understand medical videos and provide visual answers to natural language questions. We believe medical videos may provide the best possible answers to many first aid, medical emergency, and medical education questions. Toward this, we created the MedVidCL and MedVidQA datasets and introduce the tasks of Medical Video Classification (MVC) and Medical Visual Answer Localization (MVAL), two tasks that focus on cross-modal (medical language and medical video) understanding. The proposed tasks and datasets have the potential to support the development of sophisticated downstream applications that can benefit the public and medical practitioners. Our datasets consist of 6,117 fine-grained annotated videos for the MVC task and 3,010 questions and answers timestamps from 899 videos for the MVAL task. These datasets have been verified and corrected by medical informatics experts. We have also benchmarked each task with the created MedVidCL and MedVidQA datasets and propose the multimodal learning methods that set competitive baselines for future research.
18
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Corinna Cortes, Vladimir Vapnik · Machine Learning · 1995 · 40.5K citations · Full text
Convolutional Neural Networks for Sentence Classification