2020 · 16 citations · 29 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingVisual AttentionAnnotationGaze PerceptionPassage RetrievalQuestion AnsweringVisual Question AnsweringWord RecognitionGaze TrackingHighlightingNeural Question AnsweringPassage-level Question AnsweringQuestion Answering SystemsQuestion Answering Systems EvaluationQuestion UnderstandingText-based Visual Question Answering
We investigated how users evaluate passage-length answers for non-factoid questions. We conduct a study where answers were presented to users, sometimes shown with automatic word highlighting. Users were tasked with evaluating answer quality, correctness, completeness, and conciseness. Words in the answer were also annotated, both explicitly through user mark up and implicitly through user gaze data obtained from eye-tracking. Our results show that the correctness of an answer strongly depends on its completeness, conciseness is less important.
29
Andrei Broder · ACM SIGIR Forum · 2002 · 1.9K citations
Inter-Coder Agreement for Computational Linguistics
Ron Artstein, Massimo Poesio · Computational Linguistics · 2008 · 1.6K citations · Full text
Eye-tracking analysis of user behavior in WWW search
Laura Granka, Thorsten Joachims, Geri Gay · 2004 · 700 citations