Examining Question and Context Effects in Organization Survey Data Using Item Response Theory

Abstract

Organizational researchers routinely use attitudinal surveys to track organizational development and identify areas for intervention. However, seemingly trivial changes to the survey instrument, such as question wording or question order, can introduce measurement artifacts leading to differences in observed responses that are not due to actual employee attitudinal change. Traditional methods for assessing the presence of artifacts because of survey changes require additional survey administration using multiple survey forms and random assignment. However, the item response theory method illustrated in this study eliminates the need for additional data collection, offers a more rigorous design, and requires fewer organizational resources.