MSˆ2: Multi-Document Summarization of Medical Studies

TLDR

Assessing medical interventions requires time‑intensive, manual literature reviews, but NLP systems can automate or assist parts of this costly process. The authors release MSˆ2, a dataset of over 470 k documents and 20 k summaries to support multi‑document summarization of medical studies. They formulate summarization inputs and targets in both free text and structured forms while adapting a recent metric to evaluate generated summaries. MSˆ2 enables assessment and aggregation of contradictory evidence across studies, is the first large‑scale public multi‑document summarization dataset in biomedicine, and preliminary BART‑based experiments show promising results, with data and models available on GitHub.

Abstract

To assess the effectiveness of any medical intervention, researchers must conduct a time-intensive and manual literature review. NLP systems can help to automate or assist in parts of this expensive process. In support of this goal, we release MSˆ2 (Multi-Document Summarization of Medical Studies), a dataset of over 470k documents and 20K summaries derived from the scientific literature. This dataset facilitates the development of systems that can assess and aggregate contradictory evidence across multiple studies, and is the first large-scale, publicly available multi-document summarization dataset in the biomedical domain. We experiment with a summarization system based on BART, with promising early results, though significant work remains to achieve higher summarization quality. We formulate our summarization inputs and targets in both free text and structured forms and modify a recently proposed metric to assess the quality of our system's generated summaries. Data and models are available at https://github.com/allenai/ms2.