2023 · 15 citations · 44 references
ComputingAfrican American StudiesLinguisticsComputational LinguisticsNatural Language ProcessingSociolinguisticsLanguage ModelingLanguage ModelsNatural Language GenerationSecond Language AcquisitionSpeech RecognitionText GenerationBias In Language ModelsEvaluation Of Text GenerationLanguage Bias ReductionLanguage GenerationLarge Language ModelsSpeech Pattern RecognitionSpoken Language Generation
While biases disadvantaging African American Language (AAL) have been uncovered in models for tasks such as speech recognition and toxicity detection, there has been little investigation of these biases for language generation models like ChatGPT. We evaluate how well LLMs understand AAL in comparison to White Mainstream English (WME), the encouraged "standard" form of English taught in American classrooms. We measure large language model performance on two tasks: a counterpart generation task, where a model generates AAL given WME and vice versa, and a masked span prediction (MSP) task, where models predict a phrase hidden from their input. Using a novel dataset of AAL texts from a variety of regions and contexts, we present evidence of dialectal bias for six pre-trained LLMs through performance gaps on these tasks.
44
Dissecting racial bias in an algorithm used to manage the health of populations
Ziad Obermeyer, Brian W. Powers, Christine Vogeli et al. · Science · 2019 · 6.1K citations · Full text
Public Health, Health Care Management, Health Informatics +17
Critical race theory: an introduction
Choice Reviews Online · 2013 · 6.1K citations · Full text
On the Dangers of Stochastic Parrots
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major et al. · 2021 · 5.6K citations · Full text
Computing, Computer Science, English +12