2011 · 10 citations · 8 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingSyntaxData SummarizationLanguage ModelingLanguage ModelsPart-of-speech TaggingSyntactic AnalysisText SummarizationDocument ModelingDocument SummarizationLow-frequency WordsMulti-document SummarizationProbabilistic Document RepresentationStatistical SummarizationSummarizationTerm Frequency
Statistical approaches to automatic text summarization based on term frequency continue to perform on par with more complex summarization methods. To compute useful frequency statistics, however, the semantically important words must be separated from the low-content function words. The standard approach of using an a priori stopword list tends to result in both undercoverage, where syntactical words are seen as semantically relevant, and overcoverage, where words related to content are ignored. We present a generative probabilistic modeling approach to building content distributions for use with statistical multi-document summarization where the syntax words are learned directly from the data with a Hidden Markov Model and are thereby deemphasized in the term frequency statistics. This approach is compared to both a stopword-list and POS-tagging approach and our method demonstrates improved coverage on the DUC 2006 and TAC 2010 datasets using the ROUGE metric. 1
8
ROUGE: A Package for Automatic Evaluation of Summaries
Chin-Yew Lin · 2004 · 8.3K citations
The Automatic Creation of Literature Abstracts
H. P. Luhn · IBM Journal of Research and Development · 1958 · 3.2K citations
Thomas L. Griffiths, Mark Steyvers, David M. Blei et al. · 2004 · 493 citations