Student Evaluations of Teaching Effectiveness and the Leniency Hypothesis: A Literature Review.

Abstract

This review presents an overview of selected articles on the leniency hypothesis: the idea that students give higher evaluations to who grade more leniently. Such articles comprise small subset of the voluminous research on student evaluations of teaching (SETs). In this diverse literature, research methods and aims have frequently affected the outcomes and conclusions, since SETs are typically context-specific instruments whose results, in isolated instances, do not generalize well. Thus this review questions the very generalizability of the massive and often contradictory SET-related literature on the leniency hypothesis and argues that future research must be designed and carried out in light of the implicit problems existing in the majority of earlier studies. Background and Introduction to the Review Although formal instructor evaluations at the postsecondary level are also carried out by peers, administrators, and the individual themselves (as self-evaluations) (McGee, 199S), evaluations by students are the most ubiquitous. This ubiquity has resulted, according to Marsh (1984, p. 749), in student evaluations of teaching effectiveness being probably . . . the most thoroughly studied of all forms of personnel evaluation, and one of the best in terms of being supported by empirical research. Indeed, Centra (2003) reported that query to the ERIC database returned references to over 2,000 studies on student evaluations of teaching (SETs). Although dozens of new studies continue to inundate the literature every year, does the perennial popularity of this topic necessarily imply that all the scholarship and energy devoted to understanding student evaluations better is well articulated, well grounded, and worthwhile? Considering mis question from an historical perspective, for example, Marsh and Dunkin (1992) argued that numerous methodologically unsound studies on SETs were published in the 1970s, period which Centra (1993, p. 9) termed the golden age of research on student Marsh and Roche (1997, p. 1190) virtually condemned much of the literature base as nearly worthless: The voluminous literature on potential biases in· SETs is frequently atheoretical, methodologically flawed, and not based on well-articulated operational definitions of bias, thus continuing to fuel (and be fueled or fooled by) SET myths. This review considers one small sub-theme in the literature related to student evaluations of teaching effectiveness: the so-called leniency hypothesis, which attests mat instructors with more lenient grading standards receive more favourable (Wachtel, 1998, p. 200). Underscoring mis hypothesis is the notion mat can buy better evaluations by giving out higher grades (Nimmer & Stone, 1991, p. 1%). That notion, in turn, stems from what Petress (19%, p. 387) has termed the quasi economic model of education, wherein students are viewed as customers, consumers, or clients (see, for example, Bowen, 2001; Colby, Ehrlich, Beaumont, Rosner, & Stephens, 2000; Dowd, 2003; Eiszler, 2002; Muller, 1994; Stimpson, 2004). Interestingly, such conception is not new. In his 1888 inaugural address, Francis L. Patton, the twelfth president of the College of New Jersey (fate* Princeton University), remarked that college administration is business in which Trustees are partners, professors the salesmen and students the customers (Wertenbaker, 1946, p. 347). According to mis rhetoric, then, which ignores pedagogic reality (Brookfield, 1995, p. 21), SETs have become a form of customer satisfaction survey (Eiszler, p. 499). And in (common) situations where SET ratings are used by administrators to determine merit raises or to support promotion and tenure decisions, faculty are, justifiably, quite concerned about their teaching evaluations. Uses, Methods, and Analyses of Student Evaluations of Teaching Effectiveness Almost as soon as student evaluation procedures were introduced at several major universities in the U. …