Abstract
In this work, we present a method to improve the efficiency and robustness of the previous model-free Reinforcement Learning (RL) algorithms for the task of object-goal visual navigation. Despite achieving state-of-the-art results, one of the major drawbacks of those approaches is the lack of a forward model that informs the agent about the potential consequences of its actions, i.e., being model-free. In this work, we augment the model-free RL with such a forward model that can predict a representation of a future state, from the beginning of a navigation episode, if the episode were to be successful. Furthermore, in order for efficient training, we develop an algorithm to integrate a replay buffer into the model-free RL that alternates between training the policy and the forward model. We call our agent ForeSI; ForeSI is trained to imagine a future latent state that leads to success. By explicitly imagining such a state, during the navigation, our agent is able to take better actions leading to two main advantages: first, in the absence of an object detector, ForeSI presents a more robust policy, i.e., it leads to about 5% absolute improvement on the Success Rate (SR); second, when combined with an off-the-shelf object detector to help better distinguish the target object, our method leads to about 3% absolute improvement on the SR and about 2% absolute improvement on Success weighted by inverse Path Length (SPL), i.e., presents higher efficiency.
Original language | English |
---|---|
Title of host publication | Proceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022 |
Editors | Saket Anand, Ryan Farrell, Richard Souvenir, Catherine Zhao |
Place of Publication | Piscataway NJ USA |
Publisher | IEEE, Institute of Electrical and Electronics Engineers |
Pages | 3401-3410 |
Number of pages | 10 |
ISBN (Electronic) | 9781665409155 |
ISBN (Print) | 9781665409162 |
DOIs | |
Publication status | Published - 2022 |
Externally published | Yes |
Event | IEEE Winter Conference on Applications of Computer Vision 2022 - Waikoloa, United States of America Duration: 4 Jan 2022 → 8 Jan 2022 https://wacv2022.thecvf.com/home |
Conference
Conference | IEEE Winter Conference on Applications of Computer Vision 2022 |
---|---|
Abbreviated title | WACV 2022 |
Country/Territory | United States of America |
City | Waikoloa |
Period | 4/01/22 → 8/01/22 |
Internet address |
Keywords
- Analysis and Understanding
- Vision and Languages
- Vision for Robotics Multimedia Applications
- Vision Systems and Applications
- Visual Reasoning