The Sounding Board team from left to right: Professor Noah Smith, Maarten Sap, Ari Holtzman, Professor Mari Ostendorf, Elizabeth Clark, Hao Fang, Professor Yejin Choi and Hao Cheng.
The team, made up of UW graduate students in electrical engineering and computer science, is one of three in the world to make it to the finals. The inaugural $2.5 million Amazon Alexa Prize is the first of its kind for the company, asking students from around the globe to build a “socialbot” that can have intelligent conversations with Amazon’s digital assistant, Alexa, about pop culture and news and events.
The grand challenge of the Alexa Prize is to develop a socialbot with the ability to converse coherently and engagingly with humans on popular topics for 20 minutes. The team that has the highest-performing socialbot will receive the grand prize of $500,000. If the winning team also meets the 20-minute grand challenge, the team’s university will receive an additional prize of $1 million.
UW Electrical Engineering (EE) graduate student Hao Fang led the effort to develop the university’s prototype – Sounding Board. When developing the bot, the team sought a design that provides user-focused content selected from a variety of sources.
“Sounding Board is content-driven and engages the user by providing them with information that they may not already know or perspectives that they may not have heard,” Fang said. “Sounding Board is also user-driven and tracks the user’s mental state to adjust topic choice and interaction strategy.”
Applications for the Alexa Prize came in from over 100 teams across 22 countries. Sounding Board was one of 12 teams evaluated in the semifinals. The bots were evaluated based on a number of criteria, including ratings from Alexa customers and competition judges, depth and breadth of topics covered and accuracy of responses.
Fang and his teammates EE graduate student Hao Cheng and graduate students Maarten Sap, Elizabeth Clark and Ariel Holtzman from the Paul G. Allen School of Computer Science & Engineering worked with lead advisor EE Professor Mari Ostendorf and Allen School Professors Yejin Choi and Noah Smith.
For the UW team, the highly collaborative environment at the university proved a perfect testing bed for their bot. “The evolution of Sounding Board involved a lot of effort from all team members and frequent feedback from faculty advisors,” Ostendorf said. “We benefited from our experiences in data science, natural language processing, social science and system engineering. Sounding Board also benefits from the good research environment at UW, where we get a lot of useful feedback from lab mates, the UW-NLP community and both EE and CSE faculty and staff, who chatted with early versions of the system. A key resource in system development has been access to real Alexa users worldwide, which was possible through the Amazon competition. It is impossible to anticipate all the types of reactions and questions people will have, and all the different ways that even a simple yes/no question can be answered. Learning from actual user data is critical.”
From July 1 to August 15, the bots were evaluated in the semifinal round. Real customers interacted with the devices, prompting a conversation with “Alexa, let’s chat.” Customers rated the bots on a scale of one to five, assessing whether they would care to speak with the socialbot again.
In addition to Sounding Board, three other bots were selected to move on to the final round – Alquist from Czech Technical University in Prague and What’s up, Bot, a wild card pick from Heriot-Watt University in Edinburgh, Scotland.
The winning team will be announced in November at the Amazon Web Services event, AWS re:Invent.
Congratulations to team Sounding Board!