Very Large Data Bases · 2006 · 65 citations · 27 references
ComputingComputer ScienceInformation SystemsInformation RetrievalNatural Language ProcessingWeb AnalyticsInformation ExtractionSearch SystemsWeb SearchSearch Engine Result PagesSearch Engine Results PagesWeb Crawling
A search engine returned result page may contain search results that are organized into multiple dynamically generated sections in response to a user query. Furthermore, such a result page often also contains information irrelevant to the query, such as information related to the hosting site of the search engine. In this paper, we present a method to automatically generate wrappers for extracting search result records from all dynamic sections on result pages returned by search engines. This method has the following novel features: (1) it aims to explicitly identify all dynamic sections, including those that are not seen on sample result pages used to generate the wrapper, and (2) it addresses the issue of correctly differentiating sections and records. Experimental results indicate that this method is very promising. Automatic search result record extraction is critical for applications that need to interact with search engines such as automatic construction and maintenance of metasearch engines and deep Web crawling.
27
Algorithm 457: finding all cliques of an undirected graph
Coen Bron, Joep Kerbosch · Communications of the ACM · 1973 · 2.5K citations · Full text
Wrapper induction for information extraction
Nicholas Kushmerick, Daniel S. Weld · 1997 · 1K citations
Sun Wu, Udi Manber · Communications of the ACM · 1992 · 734 citations · Full text
Extracting structured data from Web pages
Arvind Arasu, Héctor García-Molina · 2003 · 701 citations