OWL: An Integrated Web-based Learning Environment

Abstract

OWL (Online Web-based Learning) is an electronic learning environment developed for General Chemistry and currently being adopted by other disciplines. Basic OWL provides a powerful, web-based electronic homework model used by thousands of students each semester. OWL’s open architecture allows for extensions that expand its scope from the delivery of straightforward electronic quizzing to the offering of a richer interactive learning environment. Such extensions include guided discovery exercises and intelligent tutors, numerous examples of which are currently being created and tested in large enrollment General Chemistry courses. In this paper we describe OWL and its use on campus. We also report on a number of OWL evaluation studies underway, including some preliminary findings from formative evaluations of OWL’s use in classes and of some of Owl’s extensions, and the first large-scale evaluation of a Stoichiometry Tutor that is integrated into OWL and that shows significant increases in student performance. Developing a New Model for Homework in Large-enrollment Classes Electronic homework has been used in the General Chemistry Program at the University of Massachusetts for 12 years. The first version was implemented using the Plato system, a commercial courseware creation package used by many universities in a variety of disciplines. The two primary General Chemistry service classes enroll between 1200 and 1400 students each semester from 35 different majors. As with any large service course, methods to encourage students to keep up with current assignments are important factors in the success of these courses. Before the introduction of electronic homework eleven years ago, General Chemistry students attended weekly recitation sections lead by faculty members, and took a quiz at the end of each section. Quizzes were graded by instructors and results were distributed one week later. While this system created the desired motivation for students to stay current with the curriculum, it was extremely labor intensive – 72 faculty contact hours each week, plus an even greater amount of TA time spent on grading. A serious pedagogical drawback was the lack of immediate feedback to the students about their work; getting a quiz back a week later provided little opportunity for students to learn from their mistakes. The Chemistry Department’s adoption of electronic homework was motivated in large part by a looming increase in the number of faculty retirements. As is the case nationally, the University was unwilling to replace these retiring faculty members on a one-for-one basis, which meant that faculty resources were under increasing pressure to cover the large contact-hour commitments required by the recitation sections. Electronic homework was seen as a way to cover these commitments with fewer faculty contact hours while maintaining the motivation factor for students to keep up with their coursework. Adoption of electronic homework afforded the opportunity to change the quiz/homework model at the same time. Because the computer grades automatically, students could take and retake “quizzes” repeatedly until they demonstrated mastery of each topic. Random selection from a large pool of questions for each quiz meant students would seldom see a question repeated, so that instructors could build in an immediate feedback cycle – after a student submitted an answer, the correct answer and a body of informative feedback was displayed that allowed the student to learn from his or her mistake immediately and apply that knowledge to the next question(s) seen. A large part of the success of this model came from the creation of a Chemistry Resource Center, a large room filled with computers that students use for their electronic homework assignments. This room was (and still is) staffed with TAs and faculty who can give help and feedback to students while they are doing their electronic homework. Instructors, TAs and students all felt that this was a better use of their time than recitation sections. The Chemistry Department was keenly interested in the results of this new model. Several of the authors carefully compared student performance in the years previous to the model’s adoption and in the years immediately after. While there was no improvement in grades, the real gains were seen in students’ affective responses – surveys showed that students overwhelmingly preferred the new approach. There were of course big gains in the cost efficiency of delivering the course. Given that the new model saved resources, was preferred by students, and did not negatively impact student performance, it was wholeheartedly embraced. Online Web-based Learning for General Chemistry The Chemistry Department began a collaboration in 1996 with the University’s Center for Computer-Based Instructional Technology (CCBIT) to replace the by-then antiquated Plato system with a Web-based version, OWL. By adopting Web technology, OWL provides a platform-independent delivery system that is available all day from any web-linked computer a student or instructor can work. It also allows developers to incorporate large off-the-shelf software components for web service, databases and middleware. The OWL system was used by the full Chemistry courses for the first time in the spring of 1997, and has been used successfully ever since. In a typical semester over 50,000 Chemistry quizzes are taken, with more than 5000 in one day during peak usage periods. 60-70% of these quizzes are taken outside the Chemistry Resource Center (i.e. in students’ rooms or other labs on campus). Student surveys administered each semester show a high degree of satisfaction with OWL, including a recognition that OWL helps students learn the material and keep up with the class. A host of similar systems have been created in recent years, such WebAssign at North Carolina State University [1], both CyberProf [2] and Mallard [3] at the University of Illinois at Urbana-Champaign, and even commercial versions such as EBSCO’s CyberExam. Most of the commercial web-based courseware development packages (e.g. Web-CT, Web-in-a-Box and TopClass) provide online quizzing capabilities. OWL, though developed independently, shares many important features with these systems. In addition, OWL has advanced features such as parameterized questions, user-defined multimedia and Java tags, and sophisticated course management tools. It is also missing some useful features found in one or more of the other systems (e.g. essay submission, audio support) that will be added soon. One feature of OWL that sets it apart from others is that, by design, its architecture is open for the addition of new learning modalities such as guided discovery modules and intelligent tutors, and for the incorporation of curriculum content materials whose use by students can be tracked by the system. These extensions to OWL are described below. OWL is now used in five Physics & Astronomy classes with almost 1000 students each semester. Like Chemistry, Physics has been able to eliminate discussion sections and TA time spent on grading. In one class, TA requirements were halved while the amount of graded homework for each student increased nine times over the previous, non-OWL semester. In this particular class, the instructor was able to measure a significant increase in student performance over the previous semester (a 6-8 point increase in mean scores for each of three midterms and one final exam), some of which can be attributed to the increased time spent on homework [1] See http://wwwassign.physics.ncsu.edu/. [2] See http://ntx2.cso.uiuc.edu/wss/services/cyberprof/. [3] See http://www.ews.uiuc.edu/Mallard/Overview/. assignments – students reported a doubling of the time spent weekly on homework assignments from one semester to the next [4]. In the coming year (1999) OWL will be used in a wide variety of departments, including Geosciences, Engineering, Microbiology, Education, Spanish & Portuguese, Art History, Mathematics, Biochemistry and Entomology. These departments have been selected by the University to participate in disseminating OWL throughout the campus. In future years OWL will be provided as a service to users from all departments, with support from the University’s academic computing facility. OWL is also being tested on other campuses. The Chemistry Department is supporting classes at UMass/Dartmouth and at Tuskegee Institute. Basic OWL runs in Windows NT and uses straightforward Common Gateway Interface programming written in C++. It uses the Netscape’s Enterprise Webserver and Microsoft’s SQL Server database program. Students and content authors (instructors, teaching assistants) can access OWL using the latest versions of Netscape Navigator and Internet Explorer. Using OWL’s Open Architecture to Integrate Interactive Learning Activities The basic OWL system has been created with funding from the University. External funding has been obtained to extend OWL from an online quizzing system to an interactive learning environment through the incorporation of such resources as guided discovery exercises and intelligent tutoring – all web-based. OWL’s open architecture allows the incorporation of these new resources by simply treating them as additional quizzes or homework assignments, all delivered over the Web. Students are assigned to work with a guided discovery exercise or intelligent tutor that is embedded in OWL, using it to engage in a learning activity much like an online laboratory. Once the student finishes the exercise, control is returned to OWL and the exercise results are stored with the student’s permanent record in OWL’s database. This allows instructors to assign active learning tasks in addition to quizzes and track students’ progress in completing them.