2009 · 58 citations · 16 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingWeb SecurityInformation ExtractionLanguage ModelingLanguage ModelsTopic ModelingWeb MiningDivergenceKullback-leibler DivergenceSpam DetectionSpam FilteringUrl AnalysisWeb Page ClassificationWeb Spam Detection
This paper applies a language model approach to different sources of information extracted from a Web page, in order to provide high quality indicators in the detection of Web Spam. Two pages linked by a hyperlink should be topically related, even though this were a weak contextual relation. For this reason we have analysed different sources of information of a Web page that belongs to the context of a link and we have applied Kullback-Leibler divergence on them for characterising the relationship between two linked pages. Moreover, we combine some of these sources of information in order to obtain richer language models. Given the different nature of internal and external links, in our study we also distinguished these types of links getting a significant improvement in classification tasks. The result is a system that improves the detection of Web Spam on two large and public datasets such as WEBSPAM-UK2006 and WEBSPAM-UK2007.
16
A Language Modeling Approach to Information Retrieval
Jay Ponte, W. Bruce Croft · ACM SIGIR Forum · 2017 · 2.5K citations · Full text
A language modeling approach to information retrieval
Jay Ponte, W. Bruce Croft · 1998 · 1.7K citations
Detecting spam web pages through content analysis
Alexandros Ntoulas, Marc Najork, Mark S. Manasse et al. · 2006 · 618 citations
Zoltán Gyöngyi, Héctor García-Molina · Adversarial Information Retrieval on the Web · 2005 · 488 citations
Carlos Castillo, Debora Donato, Aristides Gionis et al. · 2007 · 316 citations