Layered evaluation of adaptive learning systems

Abstract

This paper suggests an alternative to the traditional 'as a whole' approach of evaluating adaptive learning systems (ALS), and adaptive systems, in general.We argue that the commonly recognised models of adaptive systems can be used as a basis for a layered evaluation that offers certain benefits to the developers of ALS.Therefore, we propose the layered evaluation framework, where the success of adaptation is addressed at two distinct layers:• user modelling • adaptation decision making.We outline how layered evaluation can improve the current evaluation practice of ALS.To build a stronger case for a layered evaluation we re-visit the evaluation of the InterBook where the layered approach can provide a difference and provide an example of its use in KOD learning system.