TY - JOUR
T1 - Residual learning from demonstration
T2 - adapting DMPs for contact-rich manipulation
AU - Davchev, Todor Bozhinov
AU - Luck, Kevin Sebastian
AU - Burke, Michael
AU - Meier, Franziska
AU - Schaal, Stefan
AU - Ramamoorthy, Subramanian
N1 - Publisher Copyright:
IEEE
PY - 2022/4
Y1 - 2022/4
N2 - Manipulation skills involving contact and friction are inherent to many robotics tasks. Using the class of motor primitives for peg-in-hole like insertions, we study how robots can learn such skills. Dynamic Movement Primitives (DMP) are a popular way of extracting such policies through behaviour cloning (BC), but can struggle in the context of insertion. Policy adaptation strategies such as residual learning can help improve the overall performance of policies in the context of contact-rich manipulation. However, it is not clear how to best do this with DMPs. As result, we consider a number of possible ways for adapting a DMP formulation and propose ``residual Learning from Demonstration`` (rLfD), a framework that combines DMPs with Reinforcement Learning (RL) to learn a residual correction policy. Our evaluations suggest that applying residual learning directly in task space and operating on the full pose of the robot can significantly improve the overall performance of DMPs. We show that rLfD offers a gentle to the joints solution that improves the task success and generalisation of DMPs and enables transfer to different geometries and frictions through few-shot task adaptation. The proposed framework is evaluated on a set of tasks in which a simulated robot and a real physical robot arm have to successfully insert pegs, gears and plugs into their respective sockets. Further material and videos accompanying this paper are provided at https://sites.google.com/view/rlfd/.
AB - Manipulation skills involving contact and friction are inherent to many robotics tasks. Using the class of motor primitives for peg-in-hole like insertions, we study how robots can learn such skills. Dynamic Movement Primitives (DMP) are a popular way of extracting such policies through behaviour cloning (BC), but can struggle in the context of insertion. Policy adaptation strategies such as residual learning can help improve the overall performance of policies in the context of contact-rich manipulation. However, it is not clear how to best do this with DMPs. As result, we consider a number of possible ways for adapting a DMP formulation and propose ``residual Learning from Demonstration`` (rLfD), a framework that combines DMPs with Reinforcement Learning (RL) to learn a residual correction policy. Our evaluations suggest that applying residual learning directly in task space and operating on the full pose of the robot can significantly improve the overall performance of DMPs. We show that rLfD offers a gentle to the joints solution that improves the task success and generalisation of DMPs and enables transfer to different geometries and frictions through few-shot task adaptation. The proposed framework is evaluated on a set of tasks in which a simulated robot and a real physical robot arm have to successfully insert pegs, gears and plugs into their respective sockets. Further material and videos accompanying this paper are provided at https://sites.google.com/view/rlfd/.
KW - Adaptation models
KW - Couplings
KW - Friction
KW - Gears
KW - Learning from Demonstration
KW - Reinforcement Learning
KW - Robots
KW - Sensorimotor Learning
KW - Task analysis
KW - Trajectory
UR - http://www.scopus.com/inward/record.url?scp=85124736650&partnerID=8YFLogxK
U2 - 10.1109/LRA.2022.3150024
DO - 10.1109/LRA.2022.3150024
M3 - Article
AN - SCOPUS:85124736650
VL - 7
SP - 4488
EP - 4495
JO - IEEE Robotics and Automation Letters
JF - IEEE Robotics and Automation Letters
SN - 2377-3766
IS - 2
ER -