Proceedings of the VLDB Endowment · 2009 · 108 citations · 22 references
ComputingComputer ScienceInformation SystemsDatabasesInformation RetrievalNatural Language ProcessingInformation ExtractionQuery ProcessingRelation ExtractionRetrieval ModelsWeb SearchHtml ListsLarge-scale Web CrawlingResult MergingSchema MappingStatistical SegmentationTable Information ExtractionUnsupervised Information ExtractionWeb Information Extraction
We present the design of a system for assembling a table from a few example rows by harnessing the huge corpus of information-rich but unstructured lists on the web. We developed a totally unsupervised end to end approach which given the sample query rows --- (a) retrieves HTML lists relevant to the query from a pre-indexed crawl of web lists, (b) segments the list records and maps the segments to the query schema using a statistical model, (c) consolidates the results from multiple lists into a unified merged table, (d) and presents to the user the consolidated records ranked by their estimated membership in the target relation. The key challenges in this task include construction of new rows from very few examples, and an abundance of noisy and irrelevant lists that swamp the consolidation and ranking of rows. We propose modifications to statistical record segmentation models, and present novel consolidation and ranking techniques that can process input tables of arbitrary schema without requiring any human supervision. Experiments with Wikipedia target tables and 16 million unstructured lists show that even with just three sample rows, our system is very effective at recreating Wikipedia tables, with a mean runtime of around 20s.
22
Sunita Sarawagi · Foundations and Trends in Databases · 2008 · 706 citations
Interactive deduplication using active learning
Sunita Sarawagi, Anuradha Bhamidipaty · 2002 · 693 citations
Web data extraction based on partial tree alignment
Yanhong Zhai, Bing Liu · 2005 · 495 citations