Foreign language acquisition via artificial intelligence and extended reality: design and evaluation

Rahul R. Divekar, Jaimie Drozdal, Samuel Chabot, Yalun Zhou, Hui Su, Yue Chen, Houming Zhu, James Hendler, Jonas Braasch

Computer Assisted Language Learning · 2021 · 214 citations · 45 references

Concepts

TL;DR

AI and XR have been used to provide experiential learning akin to immersion, yet research on multimodal spoken dialogue with immersive tech and collaborative learning remains sparse, limiting L2 experiences to solo, vocabulary‑centric interactions. The study introduces the Cognitive Immersive Language Learning Environment (CILLE) to address this gap. CILLE combines AI that perceives and engages users in non‑dyadic multimodal conversations with XR that immerses students without intrusive devices, enabling naturalistic, multi‑party interactions; its effectiveness was evaluated in a seven‑week mixed‑methods study with ten university students. The study found statistically significant gains in CFL vocabulary, comprehension, and conversation skills, with student feedback and researcher observations confirming CILLE’s design and experiential impact.

Abstract

Artificial Intelligence (AI) and Extended Reality (XR) have been employed in several foreign language education applications to increase the availability of experiential learning methods akin to international immersion programs. However, research in multi-modal spoken dialogue in L2 combined with immersive technologies and collaborative learning is thin, limiting students’ experiences to solo interactions focused mostly on vocabulary and grammar in such settings. We intend to fill this gap as we present the Cognitive Immersive Language Learning Environment (CILLE). The AI in CILLE can hear, see, and understand its users and can engage with them in non-dyadic multimodal conversations. The XR offers students a feeling of being somewhere else without the use of intrusive devices and supports multi-party, multi-modal interactions. Together, AI and XR create naturalistic conversational interactions targeted towards comprehensive foreign language acquisition. We evaluate CILLE as a Chinese-as-a-foreign-language (CFL) education tool through a seven-week, mixed-methods study with university students (N = 10). Results display statistical significance and retained improvement in CFL vocabulary, comprehension, and conversation skills. Coupled with an analysis of student feedback and researcher observations, we show how CILLE is designed and experienced by students to learn CFL.

References

45