Analyzing the Linguistic Structure of Question Texts to Characterize Answerability in Quora

Suman Kalyan Maity, Aman Kharb, Animesh Mukherjee

IEEE Transactions on Computational Social Systems · 2018 · 34 citations · 31 references

Concepts

Abstract

Quora is one of the most popular community question & answer (Q&A) sites of recent times. However, with increasing question posts over time and the posts covering a wide range of topics (unlike focused Q&A sites like Stack Overflow), not all of them are getting answered. Measuring <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">answerability</i> (i.e., whether a question shall get answered or not) involves collecting expensive human judgment data that can differentiate the characteristics of an answered question from an unanswered (aka <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">open</i> ) one. Factors to judge if a question would remain open include its subjectivity, openendedness, vagueness, ambiguity, and so on. It is difficult to collect such judgments for thousands of questions, requiring automatic framework to deal the issue of answerability of questions. In this paper, we quantify: 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">user-level</i> and 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">question-level linguistic activities</i> —that can <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nicely correspond to many of the judgment factors noted earlier</i> , can be <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">easily measured for each question post</i> and that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">appropriately discriminates an answered question from an unanswered one</i> . Our central finding is that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the way users use language while writing the question text can be a very effective means to characterize answerability.</i> This characterization further helps us to predict early if a question remaining unanswered for a specific time period <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$t$ </tex-math></inline-formula> will eventually be answered or not and achieve an accuracy of 76.26% ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$t=1$ </tex-math></inline-formula> month) and 68.33% ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$t=3$ </tex-math></inline-formula> months). Notably, features representing the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">language use patterns</i> of the users are most discriminative and alone account for an accuracy of 74.18%. We also compare our method with some of the similar works <xref ref-type="bibr" rid="ref1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</xref> , <xref ref-type="bibr" rid="ref2" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[2]</xref> achieving a maximum improvement of ~39% in terms of accuracy.

References

31