2023 · 165 citations · 23 references
EducationComputer ScienceData ScienceComputer-assisted InstructionIntelligent Tutoring SystemsProgramming LanguagesEducational Software DesignFeedback GenerationProblem-solving SupportData-driven Science InstructionEducational Programming LanguagesIntelligent Tutoring Systems EvaluationLarge Language ModelsProgramming Education SupportTutoring Dialogue Systems
Computing educators face significant challenges in providing timely support to students, especially in large class settings. Large language models (LLMs) have emerged recently and show great promise for providing on-demand help at a large scale, but there are concerns that students may over-rely on the outputs produced by these models. In this paper, we introduce CodeHelp, a novel LLM-powered tool designed with guardrails to provide on-demand assistance to programming students without directly revealing solutions. We detail the design of the tool, which incorporates a number of useful features for instructors, and elaborate on the pipeline of prompting strategies we use to ensure generated outputs are suitable for students. To evaluate CodeHelp, we deployed it in a first-year computer and data science course with 52 students and collected student interactions over a 12-week period. We examine students’ usage patterns and perceptions of the tool, and we report reflections from the course instructor and a series of recommendations for classroom use. Our findings suggest that CodeHelp is well-received by students who especially value its availability and help with resolving errors, and that for instructors it is easy to deploy and complements, rather than replaces, the support that they provide to students.
23
Using thematic analysis in psychology
Virginia Braun, Victoria Clarke · Qualitative Research in Psychology · 2006 · 183.3K citations · Full text
Social Sciences, Psychology, Communication Research Methods +10