Assessing Reviewer's Performance Based on Mining Problem Localization in Peer-Review Data.

Abstract

Abstract. Current peer-review software lacks intelligence for responding to students ’ reviewing performance. As an example of an additional intelligent assessment component to such software, we propose an evaluation system that generates assessment on reviewers ’ reviewing skills regarding the issue of problem localization. We take a data mining approach, using standard supervised machine learning to build classifiers based on attributes extracted from peer-review data via Natural Language Processing techniques. Our work successfully shows it is feasible to provide intelligent support for peer-review systems to assess students ’ reviewing performance fully automatically. 1