2023 · 26 citations · 39 references
ComputingComputer ScienceComputational LinguisticsDiscourse AnalysisNatural Language ProcessingConversational InterfacesDialogue SystemsLanguage ModelingNatural Language GenerationSpeech SynthesisText GenerationChatbotsConversational AgentsConversationsDialog SystemsEvaluation Of Dialogue SystemsLanguage Processing ModelsLarge Language ModelsMulti-party Dialogue
Collecting high quality conversational data can be very expensive for most applications and infeasible for others due to privacy, ethical, or similar concerns. A promising direction to tackle this problem is to generate synthetic dialogues by prompting large language models. In this work, we use a small set of expert-written conversations as in-context examples to synthesize a social conversation dataset using prompting. We perform several thorough evaluations of our synthetic conversations compared to human-collected conversations. This includes various dimensions of conversation quality with human evaluation directly on the synthesized conversations, and interactive human evaluation of chatbots fine-tuned on the synthetically generated dataset. We additionally demonstrate that this prompting approach is generalizable to multi-party conversations, providing potential to create new synthetic data for multi-party tasks. Our synthetic multi-party conversations were rated more favorably across all measured dimensions compared to conversation excerpts sampled from a human-collected multi-party dataset.
39
Transformers: State-of-the-Art Natural Language Processing
Thomas Wolf, Lysandre Debut, Victor Sanh et al. · 2020 · 8K citations · Full text
A Diversity-Promoting Objective Function for Neural Conversation Models
Jiwei Li, Michel Galley, Chris Brockett et al. · 2016 · 2K citations · Full text
Jason Baumgartner, Savvas Zannettou, Brian Keegan et al. · Proceedings of the International AAAI Conference on Web and Social Media · 2020 · 825 citations · Full text