Human-Like Speech

Principal Investigator
Jeff Bilmes, Katrin Kirchhoff, Dani Byrd (University of Southern California), Shrikanth Narayanan (University of Southern California), Daniel Jurafsky (Stanford), Christopher Manning (Stanford)

Sponsor(s)
Office of Naval Research (ONR)

Award Period
05/01/2005 - 04/30/2008

Abstract
Computer recognition of speech is a crucial application for the Department of Defense and is a key challenge application for the scientific and engineering goals of the nation. The field has made enormous progress in the last twenty years by applying and extending the Hidden Markov Model (HMM) paradigm. But our most successful HMM systems are still too tied to specific domains such as recognizing carefully pronounced, read, or highly constrained speech. The HMM paradigm has not extended well to address accented or highly variable speech, nor has it been able to handle the crucial problem of recognition of natural conversational human-to-human speech. We believe the failure of HMMs on natural conversational speech is due to fundamental problems in this current paradigm. We propose a radically new approach to the speech-to-text problem, motivated by recent psychological results on how humans map speech to words, and grounded in powerful new statistical and machine-learning techniques. We propose to replace the HMM model at every level of speech processing, with an exciting new approach to acoustic, phonetic, and pronunciation modeling, a completely new discriminative model of word recognition, and advances in language modeling for natural conversational speech. These new model components are unified into a rich multi-stream architecture based on new probabilistic models.

Updates or corrections to this page should be sent to gheaton@u.washington.edu.