How Reliable are Measurement Scales? External Factors with Indirect Influence on Reliability Estimators

TLDR

Social science research relies on latent variable measurement with uni‑ and multi‑dimensional scales, whose validity and reliability are essential but can be compromised by external factors that affect test power and significance. This study examines whether and how external factors influence the widely used reliability estimator Cronbach’s alpha. The authors applied a bootstrapping approach to several marketing scales to assess the impact of external variables. Results indicate that respondents’ age, gender, education level, religiousness, rural/urban residence, survey type, and topic relevance can significantly alter Cronbach’s alpha values.

Abstract

In social economic researches we often need to measure non-observable, latent variables. For this we use special research instruments, with uni and multi dimensional scales designed for measuring the constructs of interest. Validity and reliability of these scales are crucial and special tests have been developed in this respect. Reliability concerns often arise, due to external factors that can influence the power and significance of such tests. Even for standardized instruments variations are possible, and they could seriously affect research results. The purpose of the present study is to investigate if and how external factors could influence a largely used reliability estimator - Cronbach Alpha. Several scales commonly used in marketing researches were tested, using a bootstrapping technique. Results show that important differences in the values of Cronbach Alpha are possible due to indirect influence from external factors - respondents' age, gender, level of study, religiousness, rural/urban living, survey type and relevance of the research subject for the participants to the survey.