Implementation testing of a hybrid symbolic/statistical multimodal architecture

Edward C. Kaiser, Philip R. Cohen

Research output: Contribution to conferencePaperpeer-review

3 Citations (Scopus)


The design and implementation of hybrid symbolic/statistical architectures is a major area of interest in current multimodal system development. Such an architecture attempts to improve multimodal recognition and disambiguation rates by using corpus-based statistics to weight the contributions from various input streams. This is in contrast to current architectures that assume independence between input streams, and combine un-weighted posterior probabilities simply by taking their cross product. Recently a Members, Teams, Committee (MTC) approach for statistically hybridizing the Quickset multimodal system has been put forward on the basis of strong empirical results in an offline analysis. MTC uses small-dimensional input streams as Members, which in turn are input into various Teams where their conditional weights are trained. The Committee then extracts a decision from the output of the Teams. This paper discusses a fully implemented regression test of MTC within Quickset, and our modification of the approach to use more specific training features. We report a relative decrease in multimodal error rate of 30%.

Original languageEnglish
Number of pages4
Publication statusPublished - 1 Jan 2002
Externally publishedYes
Event7th International Conference on Spoken Language Processing, ICSLP 2002 - Denver, United States of America
Duration: 16 Sept 200220 Sept 2002


Conference7th International Conference on Spoken Language Processing, ICSLP 2002
Country/TerritoryUnited States of America

Cite this