ComputingComputer ScienceData MiningInformation RetrievalWeb AnalyticsDocument ClusteringText ClusteringText Similarity AnalysisUnsupervised LearningWeb MiningClusteringClustering Of Web DocumentsUnstructured Data ExtractionUnsupervised ClusteringWeb Content MiningWeb Page ClusteringWeb Structure MiningWrapper Induction
An unsupervised clustering of the webpages on a website is a primary requirement for most wrapper induction and automated data extraction methods. Since page content can vary drastically across pages of one cluster (e.g., all product pages on amazon.com), traditional clustering methods typically use some distance function between the DOM trees representing a pair of webpages. However, without knowing which portions of the DOM tree are "important," such distance functions might discriminate between similar pages based on trivial features (e.g., differing number of reviews on two product pages), or club together distinct types of pages based on superficial features present in the DOM trees of both (e.g., matching footer/copyright), leading to poor clustering performance.
21
Lawrence J. Hubert, Phipps Arabie · Journal of Classification · 1985 · 7.7K citations
Rank aggregation methods for the Web
Cynthia Dwork, Ravi Kumar, Moni Naor et al. · 2001 · 1.8K citations
Syntactic clustering of the Web
Andrei Broder, S. Glassman, Mark S. Manasse et al. · Computer Networks and ISDN Systems · 1997 · 1.4K citations
Wrapper induction for information extraction
Nicholas Kushmerick, Daniel S. Weld · 1997 · 1K citations