Cybernetics and Information Technologies · 2021 · 12 citations · 16 references
ComputingComputer ScienceInformation RetrievalNatural Language ProcessingSemanticsSemantic SearchText Similarity AnalysisCosine SimilarityDocument SimilarityFocused CrawlingHybrid Matching SystemsSemantic SimilaritySemantic Similarity MatchingSimilarity SearchTopic Relevance
Abstract Topic precise crawler is a special purpose web crawler, which downloads appropriate web pages analogous to a particular topic by measuring cosine similarity or semantic similarity score. The cosine based similarity measure displays inaccurate relevance score, if topic term does not directly occur in the web page. The semantic-based similarity measure provides the precise relevance score, even if the synonyms of the given topic occur in the web page. The unavailability of the topic in the ontology produces inaccurate relevance score by the semantic focused crawlers. This paper overcomes these glitches with a hybrid string-matching algorithm by combining the semantic similarity-based measure with the probabilistic similarity-based measure. The experimental results revealed that this algorithm increased the efficiency of the focused web crawlers and achieved better Harvest Rate (HR), Precision (P) and Irrelevance Ratio (IR) than the existing web focused crawlers achieve.
16
A vector space model for automatic indexing
Gerard Salton, Anita M.-Y. Wong, Chul‐Su Yang · Communications of the ACM · 1975 · 7.4K citations · Full text
Breadth-first crawling yields high-quality pages
Marc Najork, Janet L. Wiener · 2001 · 266 citations