Assessing evolutionary terrain generation methods for curriculum reinforcement learning

David Howard, Humphrey Munn, Davide Dolcetti, Josh Kannemeyer, Nicole Robinson

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

1 Citation (Scopus)

Abstract

Curriculum learning allows complex tasks to be mastered via incremental progression over 'stepping stone' goals towards a final desired behaviour. Typical implementations learn locomotion policies for challenging environments through gradual complexification of a terrain mesh generated through a parameterised noise function. To date, researchers have predominantly generated terrains from a limited range of noise functions, and the effect of the generator on the learning process is underrepresented in the literature. We compare popular noise-based terrain generators to two indirect encodings, CPPN and GAN. To allow direct comparison between both direct and indirect representations, we assess the impact of a range of representation-agnostic MAP-Elites feature descriptors that compute metrics directly from the generated terrain meshes. Next, performance and coverage are assessed when training a humanoid robot in a physics simulator using the PPO algorithm. Results describe key differences between the generators that inform their use in curriculum learning, and present a range of useful feature descriptors for uptake by the community.

Original languageEnglish
Title of host publicationProceedings of the 2022 Genetic and Evolutionary Computation Conference
EditorsJonathan Fieldsend
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages377-384
Number of pages8
ISBN (Electronic)9781450392372
DOIs
Publication statusPublished - Jul 2022
EventThe Genetic and Evolutionary Computation Conference 2022 - Online, Boston, United States of America
Duration: 9 Jul 202213 Jul 2022
https://dl.acm.org/doi/proceedings/10.1145/3520304 (Proceedings)
https://gecco-2022.sigevo.org/HomePage (Website)

Conference

ConferenceThe Genetic and Evolutionary Computation Conference 2022
Abbreviated titleGECCO 2022
Country/TerritoryUnited States of America
CityBoston
Period9/07/2213/07/22
Internet address

Keywords

  • CPPN
  • curriculum learning
  • GAN
  • procedural content generation
  • quality-diversity
  • reinforcement learning
  • representations

Cite this