2024 · 14 citations · 17 references
ComputingAutonomous SystemsComputer ScienceArtificial IntelligenceHuman-computer InteractionInformation RetrievalEvaluation Of Retrieval SystemsLanguage ModelingRetrieval ModelsSearch SystemsUser ModelingWeb SearchHuman IntelligenceLarge Language ModelsRetrieval SystemsSearch Task ModelingUser Behavior ModelingUser Search BehaviorUser SimulationUser-centric Evaluation
Due to the advantages in the cost-efficiency and reproducibility, user simulation has become a promising solution to the user-centric evaluation of information retrieval systems. Nonetheless, accurately simulating user search behaviors has long been a challenge, because users' actions in search are highly complex and driven by intricate cognitive processes such as learning, reasoning, and planning. Recently, Large Language Models (LLMs) have demonstrated remarked potential in simulating human-level intelligence and have been used in building autonomous agents for various tasks. However, the potential of using LLMs in simulating search behaviors has not yet been fully explored. In this paper, we introduce a LLM-based user search behavior simulator, USimAgent. The proposed simulator can simulate users' querying, clicking, and stopping behaviors during search, and thus, is capable of generating complete search sessions for specific search tasks. Empirical investigation on a real user behavior dataset shows that the proposed simulator outperforms existing methods in query generation and is comparable to traditional methods in predicting user clicks and stopping behaviors. These results not only validate the effectiveness of using LLMs for user simulation but also shed light on the development of a more robust and generic user simulators.
17
An experimental comparison of click position-bias models
Nick Craswell, Onno Zoeter, Michael Taylor et al. · 2008 · 955 citations
A dynamic bayesian network click model for web search ranking
Olivier Chapelle, Ya Zhang · 2009 · 543 citations
Efficient multiple-click models in web search
Fan Guo, Chao Liu, Yi Min Wang · 2009 · 283 citations