The concept of knowledge and how to measure it

TLDR

Knowledge is commonly defined as a true, justified belief, and its measurement has traditionally relied only on answer correctness, treating correct or incorrect responses as simple indicators of knowing or not knowing while overlooking the test‑taker's certainty about those answers. The paper argues that expanding the definition of knowledge to include test‑taker certainty can alleviate serious deficiencies in traditional measurement methods, and discusses its historical roots and practical benefits. The authors describe an epistemetric method that lets respondents rate their certainty about each answer’s correctness. Computer analysis of answers and self‑assessed certainty yields multidimensional knowledge scores that address deficiencies in current assessment and achievement tests.

Abstract

Knowledge is often defined as a belief that is true and justified. This definition has led to its measurement by methods that rely solely on the correctness of answers. A correct or incorrect answer is interpreted to mean simply that a person knows or does not know something. Such methods of measurement have serious deficiencies that can be alleviated by expanding the definition of knowledge to include the test‐taker's certainty. The person's certainty about the answers on a test captures important, but now neglected, dimensions of knowledge. Historical roots of certainty as an essential component of knowledge, and some practical benefits of including it, are discussed. An epistemetric method is described which allows people to indicate “How sure are you?” about the correctness of each of their answers. A computer analysis of the person's answers and self‐assessment certainty responses provides multidimensional scores about a person's knowledge that remedy some deficiencies of knowledge assessment and achievement tests now employed.