An Exploratory Study of Student Use and Perceived Accuracy of Faculty Evaluations on RateMyProfessors.Com

Abstract

EXECUTIVE SUMMARY The website RateMyProfessors.com, established in 1999, has provided students with a means of providing publically available feedback on courses and faculty at institutions of higher education in the United States, Canada, England, Scotland and Wales (www.RateMyProfessors.com, 2008a). Due to the increased popularity of the website, academicians have investigated various issues of the online rating site. However, these studies did not answer several basic questions. In the present study, the authors seek to address the following five areas: student awareness / usage of the site, how they use the site, who do they rate, basis for their ratings, and perceived accuracy of the ratings on the site. The results are based on a sample of 152 undergraduate business students at a small regional university in the Southeast United States. Keywords: Evaluations; Faculty Evaluations; Student Evaluations; RateMyProfessor; Faculty Ratings INTRODUCTION Beginning in the late 1990s, several websites were established to provide individuals with public access to evaluations of faculty members in institutions of higher education. To date, there are five such sites operating with the most popular being RateMyProfessors.Com (hereafter referred to as RMP). According to information contained on the RMP website, over 6.8 million ratings have been submitted by students; over 1 million faculty members have been rated from over 6000 institutions (www.RateMyProfessors.com, 2008a). Anyone can access the site, search for ratings (or postings) of a faculty member and read the ratings. Individuals may search ratings by course title or subject, by institution and/or by faculty. Individuals can post a rating for a faculty member by becoming a registered user or anonymously (as a guest). Individuals rate a faculty member on a 1 - 5 point Likert scale in 4 categories: Easiness (difficulty of professor or course material); Helpfulness (professor's helpfulness and approachability); Clarity (professors' organization, time management skills and how well they teach the course material); and Rater Interest (class recommended to take before graduation) (www.RateMyProfessors.com, 2008b). An overall rating (Good Quality, Average Quality, Poor Quality) is provided based on the average scores on Helpfulness and Clarity. Additionally, individuals may elect to post comments and indicate how hot a faculty member is as indicated by a 'chili pepper' rating (www.RateMyProfessors.com, 2008b). Students have both formal and informal networks whereby they disseminate information about a faculty member to their peers; however, some students (particularly those who are new at the institution) may not have access to these networks. Most faculty evaluations are confidential, are not published by the institution, and are not available to students for review. The popularity of this site appears to be driven by the free access to faculty evaluations. However, several question the reliability and validity of the information contained on the site (cf., Jaschik, 2008; Silva, Silva, Quinn, Draper, Cover & Munoff, 2008; Otto, Sanford & Ross, 2008; Lawson & Stephenson, 2005). For example, anyone can post an evaluation of a faculty member - whether he took the course or not. Also, a person could post multiple evaluations for the same faculty member to bias his overall rating. For that matter, a faculty member with low evaluations could post evaluations to improve his overall rating (Montell, 2006). Finally the criteria used to create die ratings (two items very broadly defined) has been questioned. LITERATURE REVIEW The literature reviewed pertains to those articles which directly relate to the RMP website and/or the other sites offering a similar service (PassCollege.com, ProfessorPerformance.com, RatingsOnline.com and Reviewum.com). The authors did not include other literature as it relates to student evaluations - whether students are able to objectively evaluate faculty members, items used to evaluate faculty teaching effectiveness, etc. …