Incremental estimation of users' expertise level

Pamela Carreno-Medrano; Abhinav Dahiya; Stephen L. Smith; Dana Kulic

doi:10.1109/RO-MAN46459.2019.8956320

Incremental estimation of users' expertise level

Pamela Carreno-Medrano, Abhinav Dahiya, Stephen L. Smith, Dana Kulic

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

5 Citations (Scopus)

Abstract

Estimating a user's expertise level based on observations of their actions will result in better human-robot collaboration, by enabling the robot to adjust its behaviour and the assistance it provides according to the skills of the particular user it's interacting with. This paper details an approach to incrementally and continually estimate the expertise of a user whose goal is to optimally complete a given task. The user's expertise level, here represented as a scalar parameter, is estimated by evaluating how far their actions are from optimal. The proposed approach was tested using data from an online study where participants were asked to complete various instances of a simulated kitting task. An optimal planner was used to estimate the 'goodness' of all available actions at any given task state. We found that our expertise level estimates correlate strongly with observed after-task performance metrics and that it is possible to differentiate novices from experts after observing, on average, 33% of the errors made by the novices.

Original language	English
Title of host publication	2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019)
Editors	John-John Cabibihan, Mary Anne Williams, T. Asokan, Laxmidhar Behera
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	715-722
Number of pages	8
ISBN (Electronic)	9781728126227
ISBN (Print)	9781728126234
DOIs	https://doi.org/10.1109/RO-MAN46459.2019.8956320
Publication status	Published - 2019
Externally published	Yes
Event	IEEE/RSJ International Symposium on Robot and Human Interactive Communication 2019 - New Delhi, India Duration: 14 Oct 2019 → 18 Oct 2019 Conference number: 28th https://ro-man2019.org/ https://ieeexplore.ieee.org/xpl/conhome/8951224/proceeding (Proceedings)

Conference

Conference	IEEE/RSJ International Symposium on Robot and Human Interactive Communication 2019
Abbreviated title	RO-MAN 2019
Country/Territory	India
City	New Delhi
Period	14/10/19 → 18/10/19
Internet address	https://ro-man2019.org/ https://ieeexplore.ieee.org/xpl/conhome/8951224/proceeding (Proceedings)

Keywords

expertise estimation
HRI
human profiling

Access to Document

10.1109/RO-MAN46459.2019.8956320

Cite this

Carreno-Medrano, P., Dahiya, A., Smith, S. L., & Kulic, D. (2019). Incremental estimation of users' expertise level. In J.-J. Cabibihan, M. A. Williams, T. Asokan, & L. Behera (Eds.), 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019) (pp. 715-722). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/RO-MAN46459.2019.8956320

Carreno-Medrano, Pamela ; Dahiya, Abhinav ; Smith, Stephen L. et al. / Incremental estimation of users' expertise level. 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019). editor / John-John Cabibihan ; Mary Anne Williams ; T. Asokan ; Laxmidhar Behera. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. pp. 715-722

@inproceedings{b2f555a5d3274ad1b807cb9e073b65d3,

title = "Incremental estimation of users' expertise level",

abstract = "Estimating a user's expertise level based on observations of their actions will result in better human-robot collaboration, by enabling the robot to adjust its behaviour and the assistance it provides according to the skills of the particular user it's interacting with. This paper details an approach to incrementally and continually estimate the expertise of a user whose goal is to optimally complete a given task. The user's expertise level, here represented as a scalar parameter, is estimated by evaluating how far their actions are from optimal. The proposed approach was tested using data from an online study where participants were asked to complete various instances of a simulated kitting task. An optimal planner was used to estimate the 'goodness' of all available actions at any given task state. We found that our expertise level estimates correlate strongly with observed after-task performance metrics and that it is possible to differentiate novices from experts after observing, on average, 33% of the errors made by the novices.",

keywords = "expertise estimation, HRI, human profiling",

author = "Pamela Carreno-Medrano and Abhinav Dahiya and Smith, {Stephen L.} and Dana Kulic",

year = "2019",

doi = "10.1109/RO-MAN46459.2019.8956320",

language = "English",

isbn = "9781728126234",

pages = "715--722",

editor = "John-John Cabibihan and Williams, {Mary Anne} and T. Asokan and Laxmidhar Behera",

booktitle = "2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019)",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

note = "IEEE/RSJ International Symposium on Robot and Human Interactive Communication 2019, RO-MAN 2019 ; Conference date: 14-10-2019 Through 18-10-2019",

url = "https://ro-man2019.org/, https://ieeexplore.ieee.org/xpl/conhome/8951224/proceeding",

}

Carreno-Medrano, P, Dahiya, A, Smith, SL & Kulic, D 2019, Incremental estimation of users' expertise level. in J-J Cabibihan, MA Williams, T Asokan & L Behera (eds), 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019). IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 715-722, IEEE/RSJ International Symposium on Robot and Human Interactive Communication 2019, New Delhi, India, 14/10/19. https://doi.org/10.1109/RO-MAN46459.2019.8956320

Incremental estimation of users' expertise level. / Carreno-Medrano, Pamela; Dahiya, Abhinav; Smith, Stephen L. et al.
2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019). ed. / John-John Cabibihan; Mary Anne Williams; T. Asokan; Laxmidhar Behera. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers, 2019. p. 715-722.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Incremental estimation of users' expertise level

AU - Carreno-Medrano, Pamela

AU - Dahiya, Abhinav

AU - Smith, Stephen L.

AU - Kulic, Dana

N1 - Conference code: 28th

PY - 2019

Y1 - 2019

N2 - Estimating a user's expertise level based on observations of their actions will result in better human-robot collaboration, by enabling the robot to adjust its behaviour and the assistance it provides according to the skills of the particular user it's interacting with. This paper details an approach to incrementally and continually estimate the expertise of a user whose goal is to optimally complete a given task. The user's expertise level, here represented as a scalar parameter, is estimated by evaluating how far their actions are from optimal. The proposed approach was tested using data from an online study where participants were asked to complete various instances of a simulated kitting task. An optimal planner was used to estimate the 'goodness' of all available actions at any given task state. We found that our expertise level estimates correlate strongly with observed after-task performance metrics and that it is possible to differentiate novices from experts after observing, on average, 33% of the errors made by the novices.

AB - Estimating a user's expertise level based on observations of their actions will result in better human-robot collaboration, by enabling the robot to adjust its behaviour and the assistance it provides according to the skills of the particular user it's interacting with. This paper details an approach to incrementally and continually estimate the expertise of a user whose goal is to optimally complete a given task. The user's expertise level, here represented as a scalar parameter, is estimated by evaluating how far their actions are from optimal. The proposed approach was tested using data from an online study where participants were asked to complete various instances of a simulated kitting task. An optimal planner was used to estimate the 'goodness' of all available actions at any given task state. We found that our expertise level estimates correlate strongly with observed after-task performance metrics and that it is possible to differentiate novices from experts after observing, on average, 33% of the errors made by the novices.

KW - expertise estimation

KW - HRI

KW - human profiling

UR - http://www.scopus.com/inward/record.url?scp=85078851479&partnerID=8YFLogxK

U2 - 10.1109/RO-MAN46459.2019.8956320

DO - 10.1109/RO-MAN46459.2019.8956320

M3 - Conference Paper

AN - SCOPUS:85078851479

SN - 9781728126234

SP - 715

EP - 722

BT - 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019)

A2 - Cabibihan, John-John

A2 - Williams, Mary Anne

A2 - Asokan, T.

A2 - Behera, Laxmidhar

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

T2 - IEEE/RSJ International Symposium on Robot and Human Interactive Communication 2019

Y2 - 14 October 2019 through 18 October 2019

ER -

Carreno-Medrano P, Dahiya A, Smith SL, Kulic D. Incremental estimation of users' expertise level. In Cabibihan JJ, Williams MA, Asokan T, Behera L, editors, 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019). Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2019. p. 715-722 doi: 10.1109/RO-MAN46459.2019.8956320

Incremental estimation of users' expertise level

Abstract

Conference

Keywords

Access to Document

Other files and links

Cite this