"I See You!": A design framework for interface cues about agent visual perception from a thematic analysis of videogames

Matthew Rueben; Matthew R. Horrocks; Jennifer Eleanor Martinez; Michelle V. Cormier; Nicolas Lalone; Marlena Fraune; Phoebe O. Toups Dugas

doi:10.1145/3491102.3517699

"I See You!": A design framework for interface cues about agent visual perception from a thematic analysis of videogames

Matthew Rueben, Matthew R. Horrocks, Jennifer Eleanor Martinez, Michelle V. Cormier, Nicolas Lalone, Marlena Fraune, Phoebe O. Toups Dugas

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

3 Citations (Scopus)

Abstract

As artificial agents proliferate, there will be more and more situations in which they must communicate their capabilities to humans, including what they can "see."Artificial agents have existed for decades in the form of computer-controlled agents in videogames. We analyze videogames in order to not only inspire the design of better agents, but to stop agent designers from replicating research that has already been theorized, designed, and tested in-depth. We present a qualitative thematic analysis of sight cues in videogames and develop a framework to support human-agent interaction design. The framework identifies the different locations and stimulus types - both visualizations and sonifications - available to designers and the types of information they can convey as sight cues. Insights from several other cue properties are also presented. We close with suggestions for implementing such cues with existing technologies to improve the safety, privacy, and efficiency of human-agent interactions.

Original language	English
Title of host publication	Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
Editors	Steven Drucker, Julie Williamson, Koji Yatani
Place of Publication	New York NY USA
Publisher	Association for Computing Machinery (ACM)
Number of pages	22
ISBN (Electronic)	9781450391573
DOIs	https://doi.org/10.1145/3491102.3517699
Publication status	Published - Apr 2022
Externally published	Yes
Event	International Conference on Human Factors in Computing Systems 2022 - New Orleans, United States of America Duration: 29 Apr 2022 → 5 May 2022 Conference number: 40th https://dl.acm.org/doi/proceedings/10.1145/3491102 (ACM Digital Library - proceedings) https://chi2022.acm.org/ (Website) https://dl.acm.org/doi/proceedings/10.1145/3491101 (Extended Abstracts)

Conference

Conference	International Conference on Human Factors in Computing Systems 2022
Abbreviated title	CHI 2022
Country/Territory	United States of America
City	New Orleans
Period	29/04/22 → 5/05/22
Internet address	https://dl.acm.org/doi/proceedings/10.1145/3491102 (ACM Digital Library - proceedings) https://chi2022.acm.org/ (Website) https://dl.acm.org/doi/proceedings/10.1145/3491101 (Extended Abstracts)

Keywords

human-robot interaction
qualitative methodologies
sight cues
thematic analysis
videogames

Access to Document

10.1145/3491102.3517699

Cite this

Rueben, M., Horrocks, M. R., Martinez, J. E., Cormier, M. V., Lalone, N., Fraune, M., & Toups Dugas, P. O. (2022). "I See You!": A design framework for interface cues about agent visual perception from a thematic analysis of videogames. In S. Drucker, J. Williamson, & K. Yatani (Eds.), Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems Article 417 Association for Computing Machinery (ACM). https://doi.org/10.1145/3491102.3517699

Rueben, Matthew ; Horrocks, Matthew R. ; Martinez, Jennifer Eleanor et al. / "I See You!" : A design framework for interface cues about agent visual perception from a thematic analysis of videogames. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. editor / Steven Drucker ; Julie Williamson ; Koji Yatani. New York NY USA : Association for Computing Machinery (ACM), 2022.

@inproceedings{b19c3a7b9c964ea7bddc055de25ea6bd,

title = "{"}I See You!{"}: A design framework for interface cues about agent visual perception from a thematic analysis of videogames",

abstract = "As artificial agents proliferate, there will be more and more situations in which they must communicate their capabilities to humans, including what they can {"}see.{"}Artificial agents have existed for decades in the form of computer-controlled agents in videogames. We analyze videogames in order to not only inspire the design of better agents, but to stop agent designers from replicating research that has already been theorized, designed, and tested in-depth. We present a qualitative thematic analysis of sight cues in videogames and develop a framework to support human-agent interaction design. The framework identifies the different locations and stimulus types - both visualizations and sonifications - available to designers and the types of information they can convey as sight cues. Insights from several other cue properties are also presented. We close with suggestions for implementing such cues with existing technologies to improve the safety, privacy, and efficiency of human-agent interactions.",

keywords = "human-robot interaction, qualitative methodologies, sight cues, thematic analysis, videogames",

author = "Matthew Rueben and Horrocks, {Matthew R.} and Martinez, {Jennifer Eleanor} and Cormier, {Michelle V.} and Nicolas Lalone and Marlena Fraune and {Toups Dugas}, {Phoebe O.}",

note = "Funding Information: This material is based upon work supported by the National Science Foundation under Grant Nos. IIS-2106380, IIS-2106402, IIS-1651532, 1950121, and 1849591; the US Department of Education under Grant No. P200A180005; the New Mexico Space Grant Consortium (NMSGC) under NASA grant No. 80NSSC20M0034; and the New Mexico State University Ofce of the Vice President of Research and Graduate School (NMSU VPRGS). Publisher Copyright: {\textcopyright} 2022 ACM.; International Conference on Human Factors in Computing Systems 2022, CHI 2022 ; Conference date: 29-04-2022 Through 05-05-2022",

year = "2022",

month = apr,

doi = "10.1145/3491102.3517699",

language = "English",

editor = "Steven Drucker and Julie Williamson and Koji Yatani",

booktitle = "Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems",

publisher = "Association for Computing Machinery (ACM)",

address = "United States of America",

url = "https://dl.acm.org/doi/proceedings/10.1145/3491102, https://chi2022.acm.org/, https://dl.acm.org/doi/proceedings/10.1145/3491101",

}

Rueben, M, Horrocks, MR, Martinez, JE, Cormier, MV, Lalone, N, Fraune, M & Toups Dugas, PO 2022, "I See You!": A design framework for interface cues about agent visual perception from a thematic analysis of videogames. in S Drucker, J Williamson & K Yatani (eds), Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems., 417, Association for Computing Machinery (ACM), New York NY USA, International Conference on Human Factors in Computing Systems 2022, New Orleans, Louisiana, United States of America, 29/04/22. https://doi.org/10.1145/3491102.3517699

"I See You!": A design framework for interface cues about agent visual perception from a thematic analysis of videogames. / Rueben, Matthew; Horrocks, Matthew R.; Martinez, Jennifer Eleanor et al.
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. ed. / Steven Drucker; Julie Williamson; Koji Yatani. New York NY USA: Association for Computing Machinery (ACM), 2022. 417.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - "I See You!"

T2 - International Conference on Human Factors in Computing Systems 2022

AU - Rueben, Matthew

AU - Horrocks, Matthew R.

AU - Martinez, Jennifer Eleanor

AU - Cormier, Michelle V.

AU - Lalone, Nicolas

AU - Fraune, Marlena

AU - Toups Dugas, Phoebe O.

N1 - Conference code: 40th

PY - 2022/4

Y1 - 2022/4

N2 - As artificial agents proliferate, there will be more and more situations in which they must communicate their capabilities to humans, including what they can "see."Artificial agents have existed for decades in the form of computer-controlled agents in videogames. We analyze videogames in order to not only inspire the design of better agents, but to stop agent designers from replicating research that has already been theorized, designed, and tested in-depth. We present a qualitative thematic analysis of sight cues in videogames and develop a framework to support human-agent interaction design. The framework identifies the different locations and stimulus types - both visualizations and sonifications - available to designers and the types of information they can convey as sight cues. Insights from several other cue properties are also presented. We close with suggestions for implementing such cues with existing technologies to improve the safety, privacy, and efficiency of human-agent interactions.

AB - As artificial agents proliferate, there will be more and more situations in which they must communicate their capabilities to humans, including what they can "see."Artificial agents have existed for decades in the form of computer-controlled agents in videogames. We analyze videogames in order to not only inspire the design of better agents, but to stop agent designers from replicating research that has already been theorized, designed, and tested in-depth. We present a qualitative thematic analysis of sight cues in videogames and develop a framework to support human-agent interaction design. The framework identifies the different locations and stimulus types - both visualizations and sonifications - available to designers and the types of information they can convey as sight cues. Insights from several other cue properties are also presented. We close with suggestions for implementing such cues with existing technologies to improve the safety, privacy, and efficiency of human-agent interactions.

KW - human-robot interaction

KW - qualitative methodologies

KW - sight cues

KW - thematic analysis

KW - videogames

UR - http://www.scopus.com/inward/record.url?scp=85130576796&partnerID=8YFLogxK

U2 - 10.1145/3491102.3517699

DO - 10.1145/3491102.3517699

M3 - Conference Paper

AN - SCOPUS:85130576796

BT - Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

A2 - Drucker, Steven

A2 - Williamson, Julie

A2 - Yatani, Koji

PB - Association for Computing Machinery (ACM)

CY - New York NY USA

Y2 - 29 April 2022 through 5 May 2022

ER -

Rueben M, Horrocks MR, Martinez JE, Cormier MV, Lalone N, Fraune M et al. "I See You!": A design framework for interface cues about agent visual perception from a thematic analysis of videogames. In Drucker S, Williamson J, Yatani K, editors, Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. New York NY USA: Association for Computing Machinery (ACM). 2022. 417 doi: 10.1145/3491102.3517699

"I See You!": A design framework for interface cues about agent visual perception from a thematic analysis of videogames

Abstract

Conference

Keywords

Access to Document

Other files and links

Cite this