A Large‐Scale Evaluation of an Intelligent Discovery World: Smithtown

Abstract

Abstract Smithtown is an intelligent tutoring system designed to enhance an individual's scientific inquiry skills as well as to provide an environment for learning principles of basic microeconomics. It was hypothesized that computer instruction on applying effective interrogative skills (e.g., changing one variable at a time while holding all else constant) would ultimately lead to the acquisition of the specific subject matter. This paper presents an evaluation of Smithtown in two studies of individual differences in learning. Experiment 1, an exploratory study, demonstrated that Smithtown fared very well when compared to traditional instruction on economics and delineated the performance indicators which separated better from worse learners in this discovery environment. Experiment 2 extended the findings from the exploratory study using a large sample of subjects (N = 530) from a different population interacting with Smithtown and showed that the performance indicators relating to hypothesis generation and testing were the most predictive of successful learning in Smithtown, accounting for considerably more of the variance in our learning criterion than a measure of general intelligence. Overall, the system performed as expected. Tutoring on scientific inquiry skills resulted in increased knowledge of microeconomics. The differentiating behaviors between more and less successful subjects were in agreement with specific behaviors relating to individual differences found in general studies on problem solving and concept formation. From an instructional perspective, the behaviors we have denoted can serve as a focal point for relevant intervention studies. From a design perspective, findings from these studies suggest modifications to intelligent tutoring systems so they may be more like the individualized teaching systems they have the potential to be. Additional informationNotes on contributorsValerie J. Shute The authors wish to acknowledge the many persons whose contributions have been invaluable to this project: Bill Alley, Jeff Blais, Jeff Bonar, Ray Christal, Kathleen Katterman, Pat Kyllonen, Alan Lesgold, Kalyani Raghavan, Wes Regian, Paul Resnick, and Dan Woltz. A special note of gratitude must be extended to the creative and diligent programmers of Smithtown: Jamie Schultz and Audrey Peterson Support for the large‐scale testing and analyses of Smithtown was provided by the Learning Abilities Measurement Program (LAMP), part of the Air Force Human Resources Laboratory. The Center for the Study of Learning is funded by the Office of Educational Research and Improvement of the U.S. Department of Education. The opinions expressed do not necessarily reflect the position or policy of either AF‐HRL or OERI and no official endorsement should be inferred