Composing diverse policies for temporally extended tasks

Daniel Angelov, Yordan Hristov, Michael Burke, Subramanian Ramamoorthy

Research output: Contribution to journalArticleResearchpeer-review

2 Citations (Scopus)


Robot control policies for temporally extended and sequenced tasks are often characterized by discontinuous switches between different local dynamics. These change-points are often exploited in hierarchical motion planning to build approximate models and to facilitate the design of local, region-specific controllers. However, it becomes combinatorially challenging to implement such a pipeline for complex temporally extended tasks, especially when the sub-controllers work on different information streams, time scales and action spaces. In this letter, we introduce a method that can automatically compose diverse policies comprising motion planning trajectories, dynamic motion primitives and neural network controllers. We introduce a global goal scoring estimator that uses local, per-motion primitive dynamics models and corresponding activation state-space sets to sequence diverse policies in a locally optimal fashion. We use expert demonstrations to convert what is typically viewed as a gradient-based learning process into a planning process without explicitly specifying pre- and post-conditions. We first illustrate the proposed framework using an MDP benchmark to showcase robustness to action and model dynamics mismatch, and then with a particularly complex physical gear assembly task, solved on a PR2 robot. We show that the proposed approach successfully discovers the optimal sequence of controllers and solves both tasks efficiently.

Original languageEnglish
Pages (from-to)2658-2665
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number2
Publication statusPublished - Apr 2020
Externally publishedYes


  • learning and adaptive systems
  • learning from demonstration
  • Motion and path planning

Cite this