Retrieval-augmented Generation across Heterogeneous Knowledge

Abstract

Retrieval-augmented generation (RAG) methods have been receiving increasing attention from the NLP community and achieved stateof-the-art performance on many NLP downstream tasks. Compared with conventional pretrained generation models, RAG methods have remarkable advantages such as easy knowledge acquisition, strong scalability, and low training cost. Although existing RAG models have been applied to various knowledge-intensive NLP tasks, such as open-domain QA and dialogue systems, most of the work has focused on retrieving unstructured text documents from Wikipedia. In this paper, I first elaborate on the current obstacles to retrieving knowledge from a single-source homogeneous corpus. Then, I demonstrate evidence from both existing literature and my experiments, and provide multiple solutions on retrieval-augmented generation methods across heterogeneous knowledge.