Modular deep Q networks for sim-to-real transfer of visuo-motor policies

Fangyi Zhang, Jürgen Leitner, Michael Milford, Peter Corke

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

13 Citations (Scopus)


While deep learning has had significant successes in computer vision thanks to the abundance of visual data, collecting sufficiently large real-world datasets for robot learning can be costly. To increase the practicality of these techniques on real robots, we propose a modular deep reinforcement learning method capable of transferring models trained in simulation to a real-world robotic task. We introduce a bottleneck between perception and control, enabling the networks to be trained independently, but then merged and fine-tuned in an end-to-end manner to further improve hand-eye coordination. On a canonical, planar visually-guided robot reaching task a fine-tuned accuracy of 1.6 pixels is achieved, a significant improvement over naive transfer (17.5 pixels), showing the potential for more complicated and broader applications. Our method provides a technique for more efficient learning and transfer of visuomotor policies for real robotic systems without relying entirely on large real-world robot datasets.

Original languageEnglish
Title of host publicationAustralasian Conference on Robotics and Automation, ACRA 2017
EditorsAlen Alempijevic, Teresa Vidal Calleja, Sarath Kodagoda
Place of PublicationSydney Australia
PublisherAustralian Robotics and Automation Association (ARAA)
Number of pages10
ISBN (Electronic)9781510860117
Publication statusPublished - 2017
Externally publishedYes
EventAustralasian Conference on Robotics and Automation 2017 - University of Technology Sydney, Sydney, Australia
Duration: 11 Dec 201713 Dec 2017

Publication series

NameAustralasian Conference on Robotics and Automation, ACRA
PublisherAustralian Robotics and Automation Association (ARAA)
ISSN (Print)1448-2053


ConferenceAustralasian Conference on Robotics and Automation 2017
Abbreviated titleACRA 2017
Internet address

Cite this