Impacts of teaching towards training gesture recognizers for human-robot interaction

Jia Chuan A. Tan, Wesley P. Chan, Nicole L. Robinson, Dana Kulic, Elizabeth A. Croft

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

The use of hand-based gestures has been proposed as an intuitive way for people to communicate with robots. Typically the set of gestures is defined by the experimenter. However, existing works do not necessarily focus on gestures that are communicative, and it is unclear whether the selected gesture are actually intuitive to users. This paper investigates whether different people inherently use similar gestures to convey the same commands to robots, and how teaching of gestures when collecting demonstrations for training recognizers can improve resulting accuracy. We conducted this work in two stages. In Stage 1, we conducted an online user study (n=190) to investigate if people use similar gestures to communicate the same set of given commands to a robot when no guidance or training was given. Results revealed large variations in the gestures used among individuals With the absences of training. Training a gesture recognizer using this dataset resulted in an accuracy of around 20%. In response to this, Stage 2 involved proposing a common set of gestures for the commands. We taught these gestures through demonstrations and collected ~ 7500 videos of gestures from study participants to train another gesture recognition model. Initial results showed improved accuracy but a number of gestures had high confusion rates. Refining our gesture set and recognition model by removing those gestures, We achieved an final accuracy of 84.1 ± 2.4%. We integrated the gesture recognition model into the ROS framework and demonstrated a use case, where a person commands a robot to perform a pick and place task using the gesture set.

Original languageEnglish
Title of host publication2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2022)
EditorsAndrea Orlandini, Carmine Recchiuto
Place of PublicationPiscatway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages618-623
Number of pages6
ISBN (Electronic)9781728188591
ISBN (Print)9781665406802
DOIs
Publication statusPublished - 2022
EventIEEE/RSJ International Symposium on Robot and Human Interactive Communication 2022 - Napoli, Italy
Duration: 29 Aug 20222 Sept 2022
Conference number: 31st
http://www.ro-man2022.org/ (Website)
https://ieeexplore.ieee.org/xpl/conhome/9900506/proceeding (Proceedings)
https://web.archive.org/web/20221007214720/http://www.ro-man2022.org/ (Website (peer review note))

Publication series

NameRO-MAN 2022 - 31st IEEE International Conference on Robot and Human Interactive Communication: Social, Asocial, and Antisocial Robots
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)1944-9445
ISSN (Electronic)1944-9437

Conference

ConferenceIEEE/RSJ International Symposium on Robot and Human Interactive Communication 2022
Abbreviated titleRO-MAN 2022
Country/TerritoryItaly
CityNapoli
Period29/08/222/09/22
Internet address

Cite this