Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure

Zheyu Zhuang; Jurgen Leitner; Robert Mahony

doi:10.1109/IROS40897.2019.8968136

Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure

Zheyu Zhuang, Jurgen Leitner, Robert Mahony

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

3 Citations (Scopus)

Abstract

Visual reaching and grasping is a fundamental problem in robotics research. This paper proposes a novel approach based on deep learning a control Lyapunov function and its derivatives by encouraging a differential constraint in addition to vanilla regression that directly regresses independent joint control inputs. A key advantage of the proposed approach is that an estimate of the value of the control Lyapunov function is available in real-time that can be used to monitor the system performance and provide a level of assurance concerning progress towards the goal. The results we obtain demonstrate that the proposed approach is more robust and more reliable than vanilla regression.

Original language	English
Title of host publication	2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019
Editors	Fumihito Arai
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	4752-4759
Number of pages	8
ISBN (Electronic)	9781728140049, 9781728140032
ISBN (Print)	9781728140056
DOIs	https://doi.org/10.1109/IROS40897.2019.8968136
Publication status	Published - 2019
Externally published	Yes
Event	IEEE/RSJ International Conference on Intelligent Robots and Systems 2019 - Macau, China Duration: 3 Nov 2019 → 8 Nov 2019 https://www.iros2019.org/ https://ieeexplore.ieee.org/xpl/conhome/8957008/proceeding (Proceedings)

Publication series

Name	IEEE International Conference on Intelligent Robots and Systems
Publisher	IEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)	2153-0858
ISSN (Electronic)	2153-0866

Conference

Conference	IEEE/RSJ International Conference on Intelligent Robots and Systems 2019
Abbreviated title	IROS 2019
Country/Territory	China
City	Macau
Period	3/11/19 → 8/11/19
Internet address	https://www.iros2019.org/ https://ieeexplore.ieee.org/xpl/conhome/8957008/proceeding (Proceedings)

Access to Document

10.1109/IROS40897.2019.8968136

Cite this

Zhuang, Z., Leitner, J., & Mahony, R. (2019). Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure. In F. Arai (Ed.), 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019 (pp. 4752-4759). (IEEE International Conference on Intelligent Robots and Systems). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/IROS40897.2019.8968136

Zhuang, Zheyu ; Leitner, Jurgen ; Mahony, Robert. / Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019. editor / Fumihito Arai. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. pp. 4752-4759 (IEEE International Conference on Intelligent Robots and Systems).

@inproceedings{9c8c139c170a4df883c11f2b91bd997d,

title = "Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure",

abstract = "Visual reaching and grasping is a fundamental problem in robotics research. This paper proposes a novel approach based on deep learning a control Lyapunov function and its derivatives by encouraging a differential constraint in addition to vanilla regression that directly regresses independent joint control inputs. A key advantage of the proposed approach is that an estimate of the value of the control Lyapunov function is available in real-time that can be used to monitor the system performance and provide a level of assurance concerning progress towards the goal. The results we obtain demonstrate that the proposed approach is more robust and more reliable than vanilla regression.",

author = "Zheyu Zhuang and Jurgen Leitner and Robert Mahony",

note = "Funding Information: This research was supported by the Australian Research Council through the “Australian Centre of Excellence for Robotic Vision” CE140100016. 1 Zheyu Zhuang, Robert Mahony are with “Australian Centre for Robotic Vision”, Research School of Engineering, The Australian National University, Canberra ACT, 2601, Australia. [email protected] 2 J{\"u}rgen Leitner is with the “Australian Centre for Robotic Vision”, Queensland University of Technology (QUT), Brisbane, Australia. Publisher Copyright: {\textcopyright} 2019 IEEE.; IEEE/RSJ International Conference on Intelligent Robots and Systems 2019, IROS 2019 ; Conference date: 03-11-2019 Through 08-11-2019",

year = "2019",

doi = "10.1109/IROS40897.2019.8968136",

language = "English",

isbn = "9781728140056",

series = "IEEE International Conference on Intelligent Robots and Systems",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

pages = "4752--4759",

editor = "Fumihito Arai",

booktitle = "2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019",

address = "United States of America",

url = "https://www.iros2019.org/, https://ieeexplore.ieee.org/xpl/conhome/8957008/proceeding",

}

Zhuang, Z, Leitner, J & Mahony, R 2019, Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure. in F Arai (ed.), 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019. IEEE International Conference on Intelligent Robots and Systems, IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 4752-4759, IEEE/RSJ International Conference on Intelligent Robots and Systems 2019, Macau, China, 3/11/19. https://doi.org/10.1109/IROS40897.2019.8968136

Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure. / Zhuang, Zheyu; Leitner, Jurgen; Mahony, Robert.
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019. ed. / Fumihito Arai. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers, 2019. p. 4752-4759 (IEEE International Conference on Intelligent Robots and Systems).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure

AU - Zhuang, Zheyu

AU - Leitner, Jurgen

AU - Mahony, Robert

N1 - Funding Information: This research was supported by the Australian Research Council through the “Australian Centre of Excellence for Robotic Vision” CE140100016. 1 Zheyu Zhuang, Robert Mahony are with “Australian Centre for Robotic Vision”, Research School of Engineering, The Australian National University, Canberra ACT, 2601, Australia. [email protected] 2 Jürgen Leitner is with the “Australian Centre for Robotic Vision”, Queensland University of Technology (QUT), Brisbane, Australia. Publisher Copyright: © 2019 IEEE.

PY - 2019

Y1 - 2019

N2 - Visual reaching and grasping is a fundamental problem in robotics research. This paper proposes a novel approach based on deep learning a control Lyapunov function and its derivatives by encouraging a differential constraint in addition to vanilla regression that directly regresses independent joint control inputs. A key advantage of the proposed approach is that an estimate of the value of the control Lyapunov function is available in real-time that can be used to monitor the system performance and provide a level of assurance concerning progress towards the goal. The results we obtain demonstrate that the proposed approach is more robust and more reliable than vanilla regression.

AB - Visual reaching and grasping is a fundamental problem in robotics research. This paper proposes a novel approach based on deep learning a control Lyapunov function and its derivatives by encouraging a differential constraint in addition to vanilla regression that directly regresses independent joint control inputs. A key advantage of the proposed approach is that an estimate of the value of the control Lyapunov function is available in real-time that can be used to monitor the system performance and provide a level of assurance concerning progress towards the goal. The results we obtain demonstrate that the proposed approach is more robust and more reliable than vanilla regression.

UR - http://www.scopus.com/inward/record.url?scp=85081160529&partnerID=8YFLogxK

U2 - 10.1109/IROS40897.2019.8968136

DO - 10.1109/IROS40897.2019.8968136

M3 - Conference Paper

AN - SCOPUS:85081160529

SN - 9781728140056

T3 - IEEE International Conference on Intelligent Robots and Systems

SP - 4752

EP - 4759

BT - 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019

A2 - Arai, Fumihito

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

T2 - IEEE/RSJ International Conference on Intelligent Robots and Systems 2019

Y2 - 3 November 2019 through 8 November 2019

ER -

Zhuang Z, Leitner J, Mahony R. Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure. In Arai F, editor, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2019. p. 4752-4759. (IEEE International Conference on Intelligent Robots and Systems). doi: 10.1109/IROS40897.2019.8968136

Learning real-time closed loop robotic reaching from monocular vision by exploiting a control Lyapunov function structure

Abstract

Publication series

Conference

Access to Document

Other files and links

Cite this