Guided curriculum learning for walking over complex terrain

Brendan Tidd, Nicolas Hudson, Akansel Cosgun

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

4 Citations (Scopus)

Abstract

Reliable bipedal walking over complex terrain is a challenging problem, using a curriculum can help learning. Curriculum learning is the idea of starting with an achievable version of a task and increasing the difficulty as a success criteria is met. We propose a 3-stage curriculum to train Deep Reinforcement Learning policies for bipedal walking over various challenging terrains. In the first stage, while applying forces from a target policy to the robot joints and base, the agent starts on an easy terrain and the terrain difficulty is gradually increased. In the second stage, the guiding forces are gradually reduced to zero. Finally, in the third stage, random perturbations with increasing magnitude are applied to the robot base, so the robustness of the policies are improved. In simulation experiments, we show that our approach is effective in learning separate walking policies for five terrain types: flat, hurdles, gaps, stairs, and steps. Moreover, we demonstrate that in the absence of human demonstrations, a simple hand designed walking trajectory is a sufficient prior to learn to traverse complex terrain types. In ablation studies, we show that taking out any one of the three stages of the curriculum degrades the learning performance.

Original languageEnglish
Title of host publicationACRA 2020 Proceedings
EditorsPaulo Borges
Place of PublicationChristchurch NZ
PublisherAustralian Robotics and Automation Association (ARAA)
Number of pages10
Publication statusPublished - 2020
EventAustralasian Conference on Robotics and Automation 2020 - CSIRO’s Queensland Centre for Advanced Technologies (QCAT), Brisbane, Australia
Duration: 8 Dec 202010 Dec 2020
https://www.araa.asn.au/conference/acra-2020/
https://www.dropbox.com/s/2wq817ubujy1u7e/ACRA2020Proceedings.zip
https://research.csiro.au/robotics/acra2020/

Publication series

NameAustralasian Conference on Robotics and Automation, ACRA
PublisherAustralian Robotics and Automation Association (ARAA)
Volume2020-December
ISSN (Print)1448-2053

Conference

ConferenceAustralasian Conference on Robotics and Automation 2020
Abbreviated titleACRA 2020
Country/TerritoryAustralia
CityBrisbane
Period8/12/2010/12/20
Internet address

Cite this