The Method Evaluation Model: A Theoretical Model for Validating Information Systems Design Methods

Abstract

Information Systems (IS) design research tends to emphasise the development of new methods (normative research), while addressing the evaluation of methods in only a limited fashion. A possible reason for this is the philosophical and methodological problems involved in validating methods (“knowledge how”) as opposed to theses (“knowledge that”). “Knowledge that” has been the major focus of scientific research, which is generally about establishing the truth of particular propositions (hypotheses). However an entirely different approach is required to validate methodological knowledge. This paper proposes a theoretical model and associated measurement instrument for evaluating IS design methods. The model is based on two previously unrelated areas of theory: the Technology Acceptance Model (TAM) from the IS success literature and Methodological Pragmatism from the philosophy of science. The resulting theoretical model combines two different but related dimensions of method “success”: actual effectiveness and adoption in practice. The model is applicable to all types of IS design methods as well as methods used in other domains. A laboratory experiment and a field experiment are conducted to test the model. The paper also presents some interesting findings about the use of undergraduate students in experimental studies.