Alexa Prize — State of the Art in Conversational AI

TLDR

The article reports on research conducted during 2017–2018 to advance conversational AI through Amazon’s Alexa Prize competition. The Alexa Prize is a $2.5 million competition that invites university teams to build 20‑minute socialbots, providing them with large‑scale real‑world conversational data from millions of Alexa users and iterative feedback for improvement. Although the 20‑minute grand challenge was not met in the first year, the competition produced several agents that advanced the state of the art, proved engaging for users, and established a baseline for the next year.

Abstract

To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a $2.5 million competition that challenges university teams to build conversational agents, or “socialbots,” that can converse coherently and engagingly with humans on popular topics for 20 minutes. The Alexa Prize offers the academic community a unique opportunity to perform research at scale with real conversational data obtained by interacting with millions of Alexa users, along with user‐provided ratings and feedback, over several months. This opportunity enables teams to effectively iterate, improve, and evaluate their socialbots throughout the competition. Eighteen teams were selected for the inaugural competition last year. To build their socialbots, the students combined state‐of‐the‐art techniques with their own novel strategies in the areas of natural language understanding and conversational AI. This article reports on the research conducted over the 2017–2018 year. While the 20‐minute grand challenge was not achieved in the first year, the competition produced several conversational agents that advanced the state of the art, that are interesting for everyday users to interact with, and that help form a baseline for the second year of the competition.