2023 · 50 citations · 21 references
ComputingComputer ScienceComputational LinguisticsKnowledge RepresentationNatural Language ProcessingLanguage ModelingLanguage ModelsNatural Language GenerationNatural Language UnderstandingQuestion AnsweringText GenerationLanguage Model PromptingLarge Language ModelsModel Performance AnalysisModel Performance EvaluationOpen-domain Question AnsweringPost-editingPromptingReasoning Chains
As large language models (LLMs) have become the norm in NLP, demonstrating good performance in generation and reasoning tasks, one of its most fatal disadvantages is the lack of factual correctness. Generating unfactual texts not only leads to lower performances but also degrades the trust and validity of their applications. Chain-of-Thought (CoT) prompting improves trust and model performance on complex reasoning tasks by generating interpretable reasoning chains, but still suffers from factuality concerns in knowledge-intensive tasks. In this paper, we propose the Verify-and-Edit framework for CoT prompting, which seeks to increase prediction factuality by post-editing reasoning chains according to external knowledge. Building on top of GPT-3, our framework lead to accuracy improvements in multiple open-domain question-answering tasks.
21
Interrater reliability: the kappa statistic
Marry L. McHugh · Biochemia Medica · 2012 · 18.6K citations · Full text
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Nils Reimers, Iryna Gurevych · 2019 · 10.9K citations · Full text
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang et al. · arXiv (Cornell University) · 2022 · 4.3K citations · Full text