2024 · 22 citations · 9 references
ComputingComputer ScienceComputational LinguisticsInformation RetrievalNatural Language ProcessingDialogue SystemsLanguage ModelingNatural Language GenerationNatural Language UnderstandingQuestion AnsweringText GenerationEnd UsersExplanation GenerationLarge Language ModelsNatural Language Explanation GenerationNatural Language ExplanationsQuestion Answering SystemsRetrieval-augmented GenerationRetrieval-augmented Language Models
Owing to their size and complexity, large language models (LLMs) hardly explain why they generate a response. This effectively reduces the trust and confidence of end users in LLM-based applications, including Retrieval Augmented Generation (RAG) for Question Answering (QA) tasks. In this work, we introduce RAG-Ex, a model- and language-agnostic explanation framework that presents approximate explanations to the users revealing why the LLMs possibly generated a piece of text as a response, given the user input. Our framework is compatible with both open-source and proprietary LLMs. We report the significance scores of the approximated explanations from our generic explainer in both English and German QA tasks and also study their correlation with the downstream performance of LLMs. In the extensive user studies, our explainer yields an F1-score of 76.9% against the end user annotations and attains almost on-par performance with model-intrinsic approaches.
9
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Nils Reimers, Iryna Gurevych · 2019 · 10.9K citations · Full text
On the Dangers of Stochastic Parrots
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major et al. · 2021 · 5.6K citations · Full text
Computing, Computer Science, English +12
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang et al. · arXiv (Cornell University) · 2022 · 4.3K citations · Full text