STOP THE (AUTOGRADER) INSANITY: Regression Penalties to Deter Autograder Overreliance

Abstract

Autograders are an invaluable tool for deploying assessments in large classes. However students sometimes rely on the autograder in place of careful thought for ways to improve to their solution. We sought to naturally encourage students to check their own solutions more, and hammer the grader less. To do this, we imposed a penalty each time a student's grade went down: we called these regression penalties. We assessed whether the introduction of these penalties resulted in less reliance on the autograder without hurting student performance.