Validation of Authentic Performance Assessment: A Process Suited for Rasch Modeling

Abstract

An evaluation was conducted in order to develop, test, and validate a method for assessing critical thinking learning outcomes as related to scientific literature evaluation in a PharmD program. A 25-item authentic assessment was created using recommended guidelines and Bloom’s hierarchical taxonomy. Construct and content validity were supported using classical test theory and Rasch modeling. A primer on using Rasch analysis is provided in conjunction with the specific results. Rasch analysis demonstrated good INFIT and OUTFIT statistics, with item difficulty ranging from -1.90 to 2.46 logits for 23 items. The KR20 for the assessment was 0.41 and item point-biserial correlations ranged from -0.05 to 0.25. Rasch modeling techniques provided information to evaluate individual item contributions not discerned by standard techniques such as KR20 or point-biserial correlations. This research supports the practical application of this type of authentic assessment and provides evidence for construct and content validity in measuring student performance.