Putting Humans in the Natural Language Processing Loop: A Survey

TLDR

Human‑in‑the‑loop NLP research is emerging, with diverse frameworks that integrate human feedback to improve models across various NLP tasks. This survey examines how to design NLP systems that learn from human feedback, outlining future directions for integrating such feedback into the development loop. The authors survey HITL NLP work from ML and HCI, summarizing recent frameworks by tasks, goals, human interactions, and feedback learning methods.

Abstract

How can we design Natural Language Processing (NLP) systems that learn from human feedback? There is a growing research body of Human-in-the-loop (HITL) NLP frameworks that continuously integrate human feedback to improve the model itself. HITL NLP research is nascent but multifarious -- solving various NLP problems, collecting diverse feedback from different people, and applying different methods to learn from collected feedback. We present a survey of HITL NLP work from both Machine Learning (ML) and Human-Computer Interaction (HCI) communities that highlights its short yet inspiring history, and thoroughly summarize recent frameworks focusing on their tasks, goals, human interactions, and feedback learning methods. Finally, we discuss future directions for integrating human feedback in the NLP development loop.