An analysis of the generative AI use as analyst in qualitative research in science education

Abstract

The article evaluates the effectiveness of generative artificial intelligence models, specifically ChatGPT 4.0 and Claude 2.0, in conducting qualitative research within the field of scientific education. By applying the Cognitive Networks Mediation Theory (CNMT) to analyze interviews from two students, it was found that Claude 2.0 surpassed ChatGPT 4.0 in recognizing cognitive mediations and distinguishing between pre- and post-test conditions. Although both models concurred on the concept of conceptual evolution, Claude 2.0 demonstrated a greater capacity for detail, notably by referencing specific interview excerpts to support its analyses upon request. In contrast, ChatGPT 4.0 exhibited difficulties in these areas, even when given additional prompts. The study concludes by acknowledging the utility of AI, particularly Claude 2.0, for qualitative research, while also emphasizing the critical role of human oversight in detecting potential deficiencies within these models.