Improving the understanding of spoken referring expressions through syntactic-semantic and contextual-phonetic error-correction

Ingrid Zukerman, Andisheh Partovi

    Research output: Contribution to journalArticleResearchpeer-review

    3 Citations (Scopus)


    Despite recent advances in automatic speech recognition, one of the main stumbling blocks to the widespread adoption of Spoken Dialogue Systems is the lack of reliability of automatic speech recognizers. In this paper, we offer a two-tier error-correction process that harnesses syntactic, semantic and pragmatic information to improve the understanding of spoken referring expressions, specifically descriptions of objects in physical spaces. A syntactic-semantic tier offers generic corrections to perceived ASR errors on the basis of syntactic expectations of a semantic model, and passes the corrected texts to a language understanding system. The output of this system, which consists of pragmatic interpretations, is then refined by a contextual-phonetic tier, which prefers interpretations that are phonetically similar to the mis-heard words. Our results, obtained on a corpus of 341 referring expressions, show that syntactic-semantic error correction significantly improves interpretation performance, and contextual-phonetic refinements yield further improvements.

    Original languageEnglish
    Pages (from-to)284-310
    Number of pages27
    JournalComputer Speech and Language
    Publication statusPublished - 1 Nov 2017


    • Contextual-phonetic model
    • Error correction
    • Physical spaces
    • Pragmatic interpretation
    • Referring expressions
    • Spoken language understanding
    • Syntactic-semantic model

    Cite this