Analyzing Task Difficulty in a Bebras Contest Using Cuttle

Abstract

Predicting the difficulty level of a task on the concepts of computer science or computational thinking, like in the Bebras Challenge, proves to be really hard. But the announced difficulty level is needed in the contest format used in many local challenges. The Dutch contest system Cuttle has a new module for analysis. This is applied to one specific contest in order to find parameters explaining task difficulty. Using quantitative methods we were able to confirm a relation between answer types and difficulty and a tendency that tasks on data, data structures and representation are better answered than tasks on algorithms and programming.