What is our effect size: Evaluating the educational influence of a web-based intelligent authoring shell

Abstract

An educational system can be interpreted as a community where students and teachers are involved in the process of learning and teaching. Present-day educational systems present their users (teachers and students) an intelligent environment in order to enhance the learning and teaching process. Specifically, intelligent tutoring systems (ITSs) are computer systems designed for support and improvement of the learning and teaching process in a freely chosen knowledge domain. The goal of ITS developers is to build such systems that will create individualized instruction to get as close as possible to the 2-sigma boundary. As acquisition of knowledge is often an expensive and time-consuming process, it is important to know whether it actually improves student performance. In this paper we present some results on the evaluation of a Web-based ITS. Within this context we measure its educational effectiveness in augmenting students' accomplishments for a particular knowledge domain using the effect size as the metrics. By so doing we determine whether and in which degree an ITS increases students performance and can thus be an adequate alternative for human tutors.