Journal of Data and Information Quality · 2018 · 22 citations · 14 references
ComputingComputer ScienceInformation RetrievalNatural Language ProcessingWeb AnalyticsEvaluation Of Retrieval SystemsRetrieval ModelsSearch SystemsWeb SearchAd Hoc RetrievalInformation Retrieval EvaluationOnline EvaluationRetrievalSearch Engine EvaluationSearch EvaluationTrec EvaluationTrec Evaluation Framework
We report on our experience with TREC OpenSearch, an online evaluation campaign that enabled researchers to evaluate their experimental retrieval methods using real users of a live website. Specifically, we focus on the task of ad hoc document retrieval within the academic search domain, and work with two search engines, CiteSeerX and SSOAR, that provide us with traffic. We describe our experimental platform, which is based on the living labs methodology, and report on the experimental results obtained. We also share our experiences, challenges, and the lessons learned from running this track in 2016 and 2017.
14
Optimizing search engines using clickthrough data
Thorsten Joachims · 2002 · 3.9K citations
Controlled experiments on the web: survey and practical guide
Ron Kohavi, Roger Longbotham, Dan Sommerfield et al. · Data Mining and Knowledge Discovery · 2008 · 683 citations · Full text
Computing, Data Science, A/b Testing +16
Online controlled experiments at large scale
Ron Kohavi, Alex Deng, Brian Frasca et al. · 2013 · 432 citations