Research Square · 2023 · 421 citations · 56 references
ComputingComputer ScienceComputational LinguisticsKnowledge RepresentationNatural Language ProcessingConversational InterfacesConversational User InterfacesDialogue SystemsLanguage ModelingLanguage ModelsNatural Language UnderstandingChatbot SystemsChatbotsConversational AgentsDialog SystemsHuman ConversationHuman-ai ConversationSocietal Relevance
<title>Abstract</title> Large language models have been demonstrated to be valuable in different fields. ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation by comprehending context and generating appropriate responses. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries, with fluent and comprehensive answers surpassing prior public chatbots in both security and usefulness. However, a comprehensive analysis of ChatGPT’s failures is lacking, which is the focus of this study. Eleven categories of failures, including reasoning, factual errors, math, coding, and bias, are presented and discussed. The risks, limitations, and societal implications of ChatGPT are also highlighted. The goal of this study is to assist researchers and developers in enhancing future language models and chatbots. Please refer to here for the list of questions.
56
A large annotated corpus for learning natural language inference
Samuel R. Bowman, Gabor Angeli, Christopher Potts et al. · 2015 · 3.5K citations · Full text
Language Models are Few-Shot Learners
T. B. Brown, Benjamin Mann, Nick Ryder et al. · arXiv (Cornell University) · 2020 · 3K citations · Full text
PaLM: Scaling Language Modeling with Pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin et al. · arXiv (Cornell University) · 2022 · 2.1K citations · Full text