IEEE Access · 2020 · 93 citations · 50 references
ComputingComputer ScienceInformation TechnologyInformation RetrievalNatural Language ProcessingWeb TechnologiesInformation ExtractionKnowledge ExtractionText MiningWeb MiningWeb SearchAutomated Data ExtractionExtraction Time OptimizationHtml ElementsInformation Extraction From TextScraping TechniqueString SearchingWeb CrawlingWeb Information Extraction
Web scraping is a process of extracting valuable and interesting text information from web pages. Most of the current studies targeting this task are mostly about automated web data extraction. In the extraction process, these studies first create a DOM tree and then access the necessary data through this tree. The construction process of this tree increases the time cost depending on the data structure of the DOM Tree. In the current web scraping literature, it is observed that time efficiency is ignored. This study proposes a novel approach, namely UzunExt, which extracts content quickly using the string methods and additional information without creating a DOM Tree. The string methods consist of the following consecutive steps: searching for a given pattern, then calculating the number of closing HTML elements for this pattern, and finally extracting content for the pattern. In the crawling process, our approach collects the additional information, including the starting position for enhancing the searching process, the number of inner tag for improving the extraction process, and tag repetition for terminating the extraction process. The string methods of this novel approach are about 60 times faster than extracting with the DOM-based method. Moreover, using these additional information improves extraction time by 2.35 times compared to using only the string methods. Furthermore, this approach can easily be adapted to other DOM-based studies/parsers in this task to enhance their time efficiencies.
50
Introduction to the Theory of Computation
Michael Sipser · ACM SIGACT News · 1996 · 2.8K citations
Three models for the description of language
Noam Chomsky · IEEE Transactions on Information Theory · 1956 · 2.8K citations
Deep learning applications and challenges in big data analytics
Maryam M. Najafabadi, Flavio Villanustre, Taghi M. Khoshgoftaar et al. · Journal Of Big Data · 2015 · 2.6K citations · Full text
A fast string searching algorithm
Robert S. Boyer, J Strother Moore · Communications of the ACM · 1977 · 2.3K citations · Full text