Extending the Role of Audit Trails: A Modular Approach

Abstract

Audit trails are records of activities within educational technology environments (ETEs). (1) A generic, yet flexible audit trail system that fulfils a wide range of auditing roles and which, through a modular approach, ensures scalability and simplifies data management has been developed. While designed specifically for use with Macromedia's Director(r) authoring software, the system's framework ensures future compatibility with many widely used script-based authoring environments. The audit trail system consists of a series of functions that create and service a master record (the audit trail), comprising any number of object records corresponding to specific activities. Five types of objects are supported by the system-timer, counter, string, numeric and boolean. A separate history function can be used to create a detailed sequential record of the timing and sequence of activities. The system includes a number of developer support tools designed to simplify its implementation within target ETEs. Three examples of implementations of the audit trail system into medical courseware are described. These examples were designed to illustrate the range and type of auditing roles that the system can fulfil and to assess its performance against a series of predefined design criteria. ********** Audit trails are electronic records of users' activities within an educational technology environment (ETE), referred to by Williams and Dodge (1993) as users' what, where, and when. Although the use of audit trails by developers and researchers is relatively commonplace, most attention has focused on the interpretation and analysis of data generated by them (Misanchuk & Schwier, 1992; Salter, 1995; Chavero, Carrasco, Rossell, & Vega, 1998), rather than how the data is actually collected. This is despite the fact that, in many cases, developers and researchers must create and implement customised and often complex data collection solutions. This article proposes a more generic and systematic approach to the design and implementation of audit trail systems. It describes a new system based on modular reusable components and explores potential applications of this system through a series of pilot implementations. PURPOSES OF AUDIT TRAILS Misanchuk and Schwier (1992) define and ascribe four primary roles to audit trails: 1. formative of instructional design--that is, aimed at optimising the performance of an ETE; 2. basic in instructional design--for example, investigating how different users interact with an ETE; 3. usage audits for unstructured or public environments--that is, determining which paths or components of an ETE are most visited by particular users; and 4. counselling and advising--that is, supporting the user's decision making within an ETE based on the actions of prior users. The first three of these roles (formative evaluation, basic research, and usage audits) are closely related. In each case, data are collected within an ETE and written to an external destination for further analysis. Differentiation of these three roles lies, therefore, not in the implementation of the audit trail, but in the analysis and further application of the data it includes. All three roles are also conceptually linked as each is concerned on some level with and evaluation. All of these roles are grouped under the domain of research and evaluation (Table 1). The fourth role of audit trails defined by Misanchuk and Schwier (1992), counselling and advising, represents one of a number of related roles of audit trails that the authors placed within a second domain-that of user support. As the name suggests, support involves using audit trail data to support activities within ETEs. Rather than being collected for external analysis, here the audit trail data is used within the ETE for the purposes of internal reporting and decision-making. …