Discovery and cataloging of deep Web sources

Abstract

With more and more information goes online, extracting and managing the information from the Internet is becoming increasingly important. While the surface Web's information is relatively easy to obtain thanks to search engines such as Google and Bing, collecting the information from the deep Web is still a challenging task and these search engines do not index information located inside the deep Web. Compared to the surface Web, the deep Web contains vast more information. In particular, building a generalized search engine that can index deep Web across all domains remains a difficult research problem. In this paper, we highlight these challenges and demonstrate via prototype implementation of a generalized deep Web discovery framework that can achieve high precision.