2024 · 33 citations · 15 references
ComputingComputer ScienceLanguage EducationNatural Language ProcessingProgramming LanguagesLanguage LearningLanguage ModelingMachine TranslationMultilingual Natural Language ProcessingMultilingualismNatural Language Processing In EducationIntroductory Computer ScienceMultilingual Text GenerationNative Language InfluenceTest Cases
Large language models (LLMs) like ChatGPT are changing computing education and may create additional barriers to those already faced by non-native English speakers (NNES) learning computing. We investigate an opportunity for a positive impact of LLMs on NNES through multilingual programming exercise generation. Following previous work with LLM exercise generation in English, we prompt OpenAI GPT-3.5 in 4 natural languages (English, Tamil, Spanish, and Vietnamese) to create introductory programming problems, sample solutions, and test cases. We evaluate these problems on their sensibility, readability, translation, sample solution accuracy, topicality, and cultural relevance. We find that problems generated in English, Spanish, and Vietnamese are largely sensible, easily understood, and accurate in their sample solutions. However, Tamil problems are mostly non-sensible and have a much lower passing test rate, indicating that the abilities of LLMs for problem generation are not generalizable across languages. Our analysis suggests that these problems could not be given verbatim to students, but with minimal effort, most errors can be fixed. We further discuss the benefits of these problems despite their flaws, and their opportunities to provide personalized and culturally relevant resources for students in their native languages.
15
GitHub Copilot AI pair programmer: Asset or Liability?
Arghavan Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam et al. · Journal of Systems and Software · 2023 · 340 citations
Programming Is Hard - Or at Least It Used to Be
Brett A. Becker, Paul Denny, James Finnie-Ansley et al. · 2023 · 330 citations · Full text