Designing speech and multimodal interactions for mobile, wearable, and pervasive applications

Cosmin Munteanu; Pourang Irani; Sharon Oviatt; Matthew Aylett; Gerald Penn; Shimei Pan; Nikhil Sharma; Frank Rudzicz; Randy Gomez; Keisuke Nakamura; Kazuhiro Nakadai

doi:10.1145/2851581.2856506

Designing speech and multimodal interactions for mobile, wearable, and pervasive applications

Cosmin Munteanu, Pourang Irani, Sharon Oviatt, Matthew Aylett, Gerald Penn, Shimei Pan, Nikhil Sharma, Frank Rudzicz, Randy Gomez, Keisuke Nakamura, Kazuhiro Nakadai

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other

7 Citations (Scopus)

Abstract

Traditional interfaces are continuously being replaced by mobile, wearable, or pervasive interfaces. Yet when it comes to the input and output modalities enabling our interactions, we have yet to fully embrace some of the most natural forms of communication and information processing that humans possess: speech, language, gestures, thoughts. Very little HCI attention has been dedicated to designing and developing spoken language and multimodal interaction techniques, especially for mobile and wearable devices. In addition to the enormous, recent engineering progress in processing such modalities, there is now sufficient evidence that many real-life applications do not require 100% accuracy of processing multimodal input to be useful, particularly if such modalities complement each other. This multidisciplinary, two-day workshop will bring together interaction designers, usability researchers, and general HCI practitioners to analyze the opportunities and directions to take in designing more natural interactions with mobile and wearable devices, and to look at how we can leverage recent advances in speech and multimodal processing.

Original language	English
Title of host publication	CHI'16 - San Jose, CA USA May 7-12
Subtitle of host publication	Extended Abstracts - The 34th Annual CHI Conference on Human Factors in Computing Systems
Editors	Cliff Lampe
Place of Publication	New York NY USA
Publisher	Association for Computing Machinery (ACM)
Pages	3612-3619
Number of pages	8
ISBN (Electronic)	9781450340823
DOIs	https://doi.org/10.1145/2851581.2856506
Publication status	Published - 2016
Externally published	Yes
Event	International Conference on Human Factors in Computing Systems 2016 - San Jose, United States of America Duration: 7 May 2016 → 12 May 2016 Conference number: 34th https://chi2016.acm.org/wp/ (Website) https://dl.acm.org/doi/proceedings/10.1145/2851581 (Proceedings)

Conference

Conference	International Conference on Human Factors in Computing Systems 2016
Abbreviated title	CHI 2016
Country/Territory	United States of America
City	San Jose
Period	7/05/16 → 12/05/16
Other
Internet address	https://chi2016.acm.org/wp/ (Website) https://dl.acm.org/doi/proceedings/10.1145/2851581 (Proceedings)

Access to Document

10.1145/2851581.2856506

Cite this

Munteanu, C., Irani, P., Oviatt, S., Aylett, M., Penn, G., Pan, S., Sharma, N., Rudzicz, F., Gomez, R., Nakamura, K., & Nakadai, K. (2016). Designing speech and multimodal interactions for mobile, wearable, and pervasive applications. In C. Lampe (Ed.), CHI'16 - San Jose, CA USA May 7-12: Extended Abstracts - The 34th Annual CHI Conference on Human Factors in Computing Systems (pp. 3612-3619). Association for Computing Machinery (ACM). https://doi.org/10.1145/2851581.2856506

Munteanu, Cosmin ; Irani, Pourang ; Oviatt, Sharon et al. / Designing speech and multimodal interactions for mobile, wearable, and pervasive applications. CHI'16 - San Jose, CA USA May 7-12: Extended Abstracts - The 34th Annual CHI Conference on Human Factors in Computing Systems. editor / Cliff Lampe. New York NY USA : Association for Computing Machinery (ACM), 2016. pp. 3612-3619

@inproceedings{7f30649047fb4208bd950b7dfd48128d,

title = "Designing speech and multimodal interactions for mobile, wearable, and pervasive applications",

abstract = "Traditional interfaces are continuously being replaced by mobile, wearable, or pervasive interfaces. Yet when it comes to the input and output modalities enabling our interactions, we have yet to fully embrace some of the most natural forms of communication and information processing that humans possess: speech, language, gestures, thoughts. Very little HCI attention has been dedicated to designing and developing spoken language and multimodal interaction techniques, especially for mobile and wearable devices. In addition to the enormous, recent engineering progress in processing such modalities, there is now sufficient evidence that many real-life applications do not require 100% accuracy of processing multimodal input to be useful, particularly if such modalities complement each other. This multidisciplinary, two-day workshop will bring together interaction designers, usability researchers, and general HCI practitioners to analyze the opportunities and directions to take in designing more natural interactions with mobile and wearable devices, and to look at how we can leverage recent advances in speech and multimodal processing.",

author = "Cosmin Munteanu and Pourang Irani and Sharon Oviatt and Matthew Aylett and Gerald Penn and Shimei Pan and Nikhil Sharma and Frank Rudzicz and Randy Gomez and Keisuke Nakamura and Kazuhiro Nakadai",

year = "2016",

doi = "10.1145/2851581.2856506",

language = "English",

pages = "3612--3619",

editor = "Lampe, {Cliff }",

booktitle = "CHI'16 - San Jose, CA USA May 7-12",

publisher = "Association for Computing Machinery (ACM)",

address = "United States of America",

note = "International Conference on Human Factors in Computing Systems 2016, CHI 2016 ; Conference date: 07-05-2016 Through 12-05-2016",

url = "https://chi2016.acm.org/wp/, https://dl.acm.org/doi/proceedings/10.1145/2851581",

}

Munteanu, C, Irani, P, Oviatt, S, Aylett, M, Penn, G, Pan, S, Sharma, N, Rudzicz, F, Gomez, R, Nakamura, K & Nakadai, K 2016, Designing speech and multimodal interactions for mobile, wearable, and pervasive applications. in C Lampe (ed.), CHI'16 - San Jose, CA USA May 7-12: Extended Abstracts - The 34th Annual CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery (ACM), New York NY USA, pp. 3612-3619, International Conference on Human Factors in Computing Systems 2016, San Jose, California, United States of America, 7/05/16. https://doi.org/10.1145/2851581.2856506

Designing speech and multimodal interactions for mobile, wearable, and pervasive applications. / Munteanu, Cosmin; Irani, Pourang; Oviatt, Sharon et al.
CHI'16 - San Jose, CA USA May 7-12: Extended Abstracts - The 34th Annual CHI Conference on Human Factors in Computing Systems. ed. / Cliff Lampe. New York NY USA: Association for Computing Machinery (ACM), 2016. p. 3612-3619.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Other

TY - GEN

T1 - Designing speech and multimodal interactions for mobile, wearable, and pervasive applications

AU - Munteanu, Cosmin

AU - Irani, Pourang

AU - Oviatt, Sharon

AU - Aylett, Matthew

AU - Penn, Gerald

AU - Pan, Shimei

AU - Sharma, Nikhil

AU - Rudzicz, Frank

AU - Gomez, Randy

AU - Nakamura, Keisuke

AU - Nakadai, Kazuhiro

N1 - Conference code: 34th

PY - 2016

Y1 - 2016

N2 - Traditional interfaces are continuously being replaced by mobile, wearable, or pervasive interfaces. Yet when it comes to the input and output modalities enabling our interactions, we have yet to fully embrace some of the most natural forms of communication and information processing that humans possess: speech, language, gestures, thoughts. Very little HCI attention has been dedicated to designing and developing spoken language and multimodal interaction techniques, especially for mobile and wearable devices. In addition to the enormous, recent engineering progress in processing such modalities, there is now sufficient evidence that many real-life applications do not require 100% accuracy of processing multimodal input to be useful, particularly if such modalities complement each other. This multidisciplinary, two-day workshop will bring together interaction designers, usability researchers, and general HCI practitioners to analyze the opportunities and directions to take in designing more natural interactions with mobile and wearable devices, and to look at how we can leverage recent advances in speech and multimodal processing.

AB - Traditional interfaces are continuously being replaced by mobile, wearable, or pervasive interfaces. Yet when it comes to the input and output modalities enabling our interactions, we have yet to fully embrace some of the most natural forms of communication and information processing that humans possess: speech, language, gestures, thoughts. Very little HCI attention has been dedicated to designing and developing spoken language and multimodal interaction techniques, especially for mobile and wearable devices. In addition to the enormous, recent engineering progress in processing such modalities, there is now sufficient evidence that many real-life applications do not require 100% accuracy of processing multimodal input to be useful, particularly if such modalities complement each other. This multidisciplinary, two-day workshop will bring together interaction designers, usability researchers, and general HCI practitioners to analyze the opportunities and directions to take in designing more natural interactions with mobile and wearable devices, and to look at how we can leverage recent advances in speech and multimodal processing.

UR - http://www.scopus.com/inward/record.url?scp=85014715014&partnerID=8YFLogxK

U2 - 10.1145/2851581.2856506

DO - 10.1145/2851581.2856506

M3 - Conference Paper

AN - SCOPUS:85014715014

SP - 3612

EP - 3619

BT - CHI'16 - San Jose, CA USA May 7-12

A2 - Lampe, Cliff

PB - Association for Computing Machinery (ACM)

CY - New York NY USA

T2 - International Conference on Human Factors in Computing Systems 2016

Y2 - 7 May 2016 through 12 May 2016

ER -

Munteanu C, Irani P, Oviatt S, Aylett M, Penn G, Pan S et al. Designing speech and multimodal interactions for mobile, wearable, and pervasive applications. In Lampe C, editor, CHI'16 - San Jose, CA USA May 7-12: Extended Abstracts - The 34th Annual CHI Conference on Human Factors in Computing Systems. New York NY USA: Association for Computing Machinery (ACM). 2016. p. 3612-3619 doi: 10.1145/2851581.2856506

Designing speech and multimodal interactions for mobile, wearable, and pervasive applications

Abstract

Conference

Access to Document

Other files and links

Cite this