Persuasion with Precision: Using Natural Language Processing to Improve Instrument Fidelity for Risk Communication Experimental Treatments

Abstract

Instrument fidelity in message testing research hinges upon how precisely messages operationalize treatment conditions. However, numerous message testing studies have unmitigated threats to validity and reliability because no established procedures exist to guide construction of message treatments. Their construction typically occurs in a black box, resulting in suspect inferential conclusions about treatment effects. Because a mixed methods approach is needed to enhance instrument fidelity in message testing research, this article contributes to the field of mixed methods research by presenting an integrated multistage procedure for constructing precise message treatments using an exploratory sequential mixed methods design. This work harnesses the power of integration through crossover analysis to improve instrument fidelity in message testing research through the use of natural language processing (NLP).