2006 · 24 citations · 16 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingText AnalyticsComputational Text AnalysisDocument AnalysisText MiningText RepresentationText SegmentationTopic ModelingEvaluation MetricsSegmentationSegmentation Evaluation MetricsWord Segmentation
We consider here the task of linear the-matic segmentation of text documents, by using features based on word distributions in the text. For this task, a typical and often implicit assumption in previous studies is that a document has just one topic and therefore many algorithms have been tested and have shown encouraging results on artificial data sets, generated by putting together parts of different documents. We show that evaluation on synthetic data is potentially misleading and fails to give an accurate evaluation of the performance on real data. Moreover, we provide a critical review of existing evaluation metrics in the literature and we propose an improved evaluation metric.
16
An algorithm for suffix stripping
Martin Porter · Program electronic library and information systems · 1980 · 8.1K citations
Statistical Models for Text Segmentation
Doug Beeferman, Adam Berger, John Lafferty · Machine Learning · 1999 · 687 citations · Full text