Longitudinal Study of Online Statics Homework as a Method to Improve Learning

Abstract

AbstractBackground - Score improvements and retention are important metrics in studying the efficacy of any learning intervention. Research studies have shown a positive correlation between time spent completing online homework assignments and student performance on final exams. Yet these studies are not only rare in the field of engineering; they are generally not longitudinal in nature.Method -This study design was three-fold. Firstly, it employed a quasi-experimental study using an online homework system, MasteringEngineering, in the statics course. Students were compared in two statics courses. In one course, students submitted written homework, and in the other course students worked and submitted homework problems online. Students received the same final exam, making direct comparisons feasible. Secondly, data were gathered from a traditional Mechanics of Material course where no online homework system was employed. Again, the same final exam was given for direct comparison. Thirdly, a concept analysis was run using Concept Assessment Tool for Statics (Steif, 2010).Results - Students who used the online system showed an improvement of 0.7 (± 0.2) in effect size on the final exam when compared to written homework. Students who used the online system scored 79 percent (SD= 8 percent; N= 69) on average on the final exam. In comparison, students prepared via written homework scored 70 percent (SD= 16 percent; N= 64) on average on the same final exam. These results held for the subsequent mechanics course where students previously prepared via online statics homework scored 79 percent (SD= 8 percent; N= 66), and students prepared via written homework scored 63 percent (SD= 18 percent; N= 79) on the same final. Furthermore, an independent instructor taught the mechanics course from the statics course, removing potential bias. Results were statistically significant. Independent of the historical analysis, another analysis was run on students in the online homework statics course. Students were given pre / post tests to measure understanding on nine concepts, with friction being the only topic yielding an inadequate learning gain (difficulty change of friction 0.04 ±0.28). In conclusion, the online homework intervention showed an improvement of 0.7 effect size when all other elements in the course remained unchanged.I. BackgroundResearch studies have generally shown a positive correlation between time spent completing homework assignments and student performance. For example, Paschal et al. (1984) found a positive effect of homework on achievement in their review of 15 studies on the subject. Cooper (1989) found positive effects in 14 of the 20 studies (70 percent) they reviewed. Cooper (1989) also found evidence that students who spend more time on homework received better grades in 43 of the 50 reviewed studies (86 percent). Keith (1982) showed that, among high school students, those with low ability were able to achieve grades commensurate with students of higher ability if they did one to three hours of homework per week.The importance of feedback and grade-dependence on homework has also been shown. Trussell and Dietz (2003) showed that test performance for electrical engineers was better for students whose homework was graded than for those whose homework was not graded (a repeated experiment did not show a statistical difference). Paschal et al. (1984) concluded that the positive effects of homework were enhanced when it was graded and also when comments were written on the assignment.The idea of web-based homework is not new, and several studies have reported some advantages to this type of learning. Flori et al. (2002) showed that students who used web-based homework for engineering dynamics scored better on exams than those who did not. Taraban et al. (2005) found that the use of online thermodynamics homework did not require excessive time from the students, resulted in improved test scores, and that the immediate feedback from the system was the primary reason that it was such an effective learning tool. …