2024 · 15 citations · 6 references
ComputingComputer ScienceEducational TechnologyArtificial IntelligenceIntelligent Tutoring SystemsNatural Language ProcessingDialogue SystemsDialogue-based TutoringLanguage ModelingNatural Language Processing In EducationNatural Language UnderstandingTutorial Dialogue SystemsConversation SimulationConversational Tutoring AgentsDialog SystemsIntelligent Tutoring DialoguesInternal MonologueLarge Language ModelsTutorial Dialogue AgentsTutoring Dialogue Systems
High-quality conversational datasets are crucial for the successful development of Intelligent Tutoring Systems (ITS) that utilize a Large Language Model (LLM) backend. Synthetic student-teacher dialogues, generated using advanced GPT-4 models, are a common strategy for creating these datasets. However, subjects like physics that entail complex calculations pose a challenge. While GPT-4 presents impressive language processing capabilities, its limitations in fundamental mathematical reasoning curtail its efficacy for such subjects. To tackle this limitation, we introduce in this paper an innovative stateful prompt design. Our design orchestrates a mock conversation where both student and tutorbot roles are simulated by GPT-4. Each student response triggers an internal monologue, or ‘code soliloquy’ in the GPT-tutorbot, which assesses whether its subsequent response would necessitate calculations. If a calculation is deemed necessary, it scripts the relevant Python code and uses the Python output to construct a response to the student. Our approach notably enhances the quality of synthetic conversation datasets, especially for subjects that are calculation-intensive. The preliminary Subject Matter Expert evaluations reveal that our Higgs model, a fine-tuned LLaMA model, effectively uses Python for computations, which significantly enhances the accuracy and computational reliability of Higgs’ responses.
6
Jeannette M. Wing · Communications of the ACM · 2006 · 7.1K citations · Full text
Open-ended Knowledge Tracing for Computer Science Education
Naiming Liu, Zichao Wang, Richard G. Baraniuk et al. · 2022 · 34 citations · Full text