ForeSI: Success-Aware Visual Navigation Agent

Mahdi Kazemi Moghaddam, Ehsan Abbasnejad, Qi Wu, Javen Qinfeng Shi, Anton Van Den Hengel

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

8 Citations (Scopus)

Abstract

In this work, we present a method to improve the efficiency and robustness of the previous model-free Reinforcement Learning (RL) algorithms for the task of object-goal visual navigation. Despite achieving state-of-the-art results, one of the major drawbacks of those approaches is the lack of a forward model that informs the agent about the potential consequences of its actions, i.e., being model-free. In this work, we augment the model-free RL with such a forward model that can predict a representation of a future state, from the beginning of a navigation episode, if the episode were to be successful. Furthermore, in order for efficient training, we develop an algorithm to integrate a replay buffer into the model-free RL that alternates between training the policy and the forward model. We call our agent ForeSI; ForeSI is trained to imagine a future latent state that leads to success. By explicitly imagining such a state, during the navigation, our agent is able to take better actions leading to two main advantages: first, in the absence of an object detector, ForeSI presents a more robust policy, i.e., it leads to about 5% absolute improvement on the Success Rate (SR); second, when combined with an off-the-shelf object detector to help better distinguish the target object, our method leads to about 3% absolute improvement on the SR and about 2% absolute improvement on Success weighted by inverse Path Length (SPL), i.e., presents higher efficiency.

Original languageEnglish
Title of host publicationProceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022
EditorsSaket Anand, Ryan Farrell, Richard Souvenir, Catherine Zhao
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages3401-3410
Number of pages10
ISBN (Electronic)9781665409155
ISBN (Print)9781665409162
DOIs
Publication statusPublished - 2022
Externally publishedYes
EventIEEE Winter Conference on Applications of Computer Vision 2022 - Waikoloa, United States of America
Duration: 4 Jan 20228 Jan 2022
https://wacv2022.thecvf.com/home

Conference

ConferenceIEEE Winter Conference on Applications of Computer Vision 2022
Abbreviated titleWACV 2022
Country/TerritoryUnited States of America
CityWaikoloa
Period4/01/228/01/22
Internet address

Keywords

  • Analysis and Understanding
  • Vision and Languages
  • Vision for Robotics Multimedia Applications
  • Vision Systems and Applications
  • Visual Reasoning

Cite this