Unscripted retargeting: reach prediction for haptic retargeting in virtual reality

Aldrich Clarence; Jarrod Knibbe; Maxime Cordeil; Michael Wybrow

doi:10.1109/VR50410.2021.00036

Unscripted retargeting: reach prediction for haptic retargeting in virtual reality

Aldrich Clarence, Jarrod Knibbe, Maxime Cordeil, Michael Wybrow

Department of Human Centred Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

28 Citations (Scopus)

Abstract

Research is exploring novel ways of adding haptics to VR. One popular technique is haptic retargeting, where real and virtual hands are decoupled to enable the reuse of physical props. However, this technique requires the system to know the users' intended interaction target, or requires additional hardware for prediction. We explore software-based reach prediction as a means of facilitating responsive, unscripted retargeting. We trained a Long Short-Term Memory network on users' reach trajectories to predict intended targets. We achieved an accuracy of 81.1 % at approximately 65% of movement. This could enable haptic retargeting during the last 35% of movement. We discuss the implications for possible physical proxy locations.

Original language	English
Title of host publication	Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021
Editors	Regis Kopper, Gerd Bruder, Pablo Figueroa, Martin Hachet, Tobias Langlotz, Luciana Nedel, Lili Wang
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	150-159
Number of pages	10
ISBN (Electronic)	9780738125565, 9781665418386
ISBN (Print)	9781665430685
DOIs	https://doi.org/10.1109/VR50410.2021.00036
Publication status	Published - 2021
Event	IEEE Conference on Virtual Reality and 3D User Interfaces 2021 - Online, Lisboa, Portugal Duration: 27 Mar 2021 → 2 Apr 2021 Conference number: 28th https://ieeexplore.ieee.org/xpl/conhome/9417603/proceeding (Proceedings) https://ieeevr.org/2021/ (Website)

Publication series

Name	Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021
Publisher	The Institute of Electrical and Electronics Engineers, Inc.
ISSN (Print)	2642-5246
ISSN (Electronic)	2642-5254

Conference

Conference	IEEE Conference on Virtual Reality and 3D User Interfaces 2021
Abbreviated title	VR 2021
Country/Territory	Portugal
City	Lisboa
Period	27/03/21 → 2/04/21
Internet address	https://ieeexplore.ieee.org/xpl/conhome/9417603/proceeding (Proceedings) https://ieeevr.org/2021/ (Website)

Keywords

Computing methodologies-Machine learning-Machine learning approaches-Neural networks
Human-centered computing-Human computer interaction (HCI)-Interaction paradigms-Virtual reality

Access to Document

10.1109/VR50410.2021.00036

Cite this

Clarence, A., Knibbe, J., Cordeil, M., & Wybrow, M. (2021). Unscripted retargeting: reach prediction for haptic retargeting in virtual reality. In R. Kopper, G. Bruder, P. Figueroa, M. Hachet, T. Langlotz, L. Nedel, & L. Wang (Eds.), Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021 (pp. 150-159). (Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/VR50410.2021.00036

Clarence, Aldrich ; Knibbe, Jarrod ; Cordeil, Maxime et al. / Unscripted retargeting : reach prediction for haptic retargeting in virtual reality. Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021. editor / Regis Kopper ; Gerd Bruder ; Pablo Figueroa ; Martin Hachet ; Tobias Langlotz ; Luciana Nedel ; Lili Wang. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2021. pp. 150-159 (Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021).

@inproceedings{2c6e224aada143af8d7d80272e7c5d4c,

title = "Unscripted retargeting: reach prediction for haptic retargeting in virtual reality",

abstract = "Research is exploring novel ways of adding haptics to VR. One popular technique is haptic retargeting, where real and virtual hands are decoupled to enable the reuse of physical props. However, this technique requires the system to know the users' intended interaction target, or requires additional hardware for prediction. We explore software-based reach prediction as a means of facilitating responsive, unscripted retargeting. We trained a Long Short-Term Memory network on users' reach trajectories to predict intended targets. We achieved an accuracy of 81.1 % at approximately 65% of movement. This could enable haptic retargeting during the last 35% of movement. We discuss the implications for possible physical proxy locations.",

keywords = "Computing methodologies-Machine learning-Machine learning approaches-Neural networks, Human-centered computing-Human computer interaction (HCI)-Interaction paradigms-Virtual reality",

author = "Aldrich Clarence and Jarrod Knibbe and Maxime Cordeil and Michael Wybrow",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.; IEEE Conference on Virtual Reality and 3D User Interfaces 2021, VR 2021 ; Conference date: 27-03-2021 Through 02-04-2021",

year = "2021",

doi = "10.1109/VR50410.2021.00036",

language = "English",

isbn = "9781665430685",

series = "Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

pages = "150--159",

editor = "Kopper, {Regis } and Bruder, {Gerd } and Figueroa, {Pablo } and Hachet, {Martin } and Langlotz, {Tobias } and Nedel, {Luciana } and Wang, {Lili }",

booktitle = "Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021",

address = "United States of America",

url = "https://ieeexplore.ieee.org/xpl/conhome/9417603/proceeding, https://ieeevr.org/2021/",

}

Clarence, A, Knibbe, J, Cordeil, M & Wybrow, M 2021, Unscripted retargeting: reach prediction for haptic retargeting in virtual reality. in R Kopper, G Bruder, P Figueroa, M Hachet, T Langlotz, L Nedel & L Wang (eds), Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021. Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021, IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 150-159, IEEE Conference on Virtual Reality and 3D User Interfaces 2021, Lisboa, Portugal, 27/03/21. https://doi.org/10.1109/VR50410.2021.00036

Unscripted retargeting: reach prediction for haptic retargeting in virtual reality. / Clarence, Aldrich; Knibbe, Jarrod; Cordeil, Maxime et al.
Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021. ed. / Regis Kopper; Gerd Bruder; Pablo Figueroa; Martin Hachet; Tobias Langlotz; Luciana Nedel; Lili Wang. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers, 2021. p. 150-159 (Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Unscripted retargeting

T2 - IEEE Conference on Virtual Reality and 3D User Interfaces 2021

AU - Clarence, Aldrich

AU - Knibbe, Jarrod

AU - Cordeil, Maxime

AU - Wybrow, Michael

N1 - Conference code: 28th

PY - 2021

Y1 - 2021

N2 - Research is exploring novel ways of adding haptics to VR. One popular technique is haptic retargeting, where real and virtual hands are decoupled to enable the reuse of physical props. However, this technique requires the system to know the users' intended interaction target, or requires additional hardware for prediction. We explore software-based reach prediction as a means of facilitating responsive, unscripted retargeting. We trained a Long Short-Term Memory network on users' reach trajectories to predict intended targets. We achieved an accuracy of 81.1 % at approximately 65% of movement. This could enable haptic retargeting during the last 35% of movement. We discuss the implications for possible physical proxy locations.

AB - Research is exploring novel ways of adding haptics to VR. One popular technique is haptic retargeting, where real and virtual hands are decoupled to enable the reuse of physical props. However, this technique requires the system to know the users' intended interaction target, or requires additional hardware for prediction. We explore software-based reach prediction as a means of facilitating responsive, unscripted retargeting. We trained a Long Short-Term Memory network on users' reach trajectories to predict intended targets. We achieved an accuracy of 81.1 % at approximately 65% of movement. This could enable haptic retargeting during the last 35% of movement. We discuss the implications for possible physical proxy locations.

KW - Computing methodologies-Machine learning-Machine learning approaches-Neural networks

KW - Human-centered computing-Human computer interaction (HCI)-Interaction paradigms-Virtual reality

UR - http://www.scopus.com/inward/record.url?scp=85106527127&partnerID=8YFLogxK

U2 - 10.1109/VR50410.2021.00036

DO - 10.1109/VR50410.2021.00036

M3 - Conference Paper

AN - SCOPUS:85106527127

SN - 9781665430685

T3 - Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021

SP - 150

EP - 159

BT - Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021

A2 - Kopper, Regis

A2 - Bruder, Gerd

A2 - Figueroa, Pablo

A2 - Hachet, Martin

A2 - Langlotz, Tobias

A2 - Nedel, Luciana

A2 - Wang, Lili

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

Y2 - 27 March 2021 through 2 April 2021

ER -

Clarence A, Knibbe J, Cordeil M, Wybrow M. Unscripted retargeting: reach prediction for haptic retargeting in virtual reality. In Kopper R, Bruder G, Figueroa P, Hachet M, Langlotz T, Nedel L, Wang L, editors, Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2021. p. 150-159. (Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2021). doi: 10.1109/VR50410.2021.00036

Unscripted retargeting: reach prediction for haptic retargeting in virtual reality

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Cite this