Look no deeper: recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation

Sourav Garg, Madhu V. Babu, Thanuja Dharmasiri, Stephen Hausler, Niko Suenderhauf, Swagat Kumar, Tom Drummond, Michael Milford

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Visual place recognition (VPR) - the act of recognizing a familiar visual place - becomes difficult when there is extreme environmental appearance change or viewpoint change. Particularly challenging is the scenario where both phenomena occur simultaneously, such as when returning for the first time along a road at night that was previously traversed during the day in the opposite direction. While such problems can be solved with panoramic sensors, humans solve this problem regularly with limited field-of-view vision and without needing to constantly turn around. In this paper, we present a new depth- and temporal-aware visual place recognition system that solves the opposing viewpoint, extreme appearance-change visual place recognition problem. Our system performs sequence-to-single frame matching by extracting depth-filtered keypoints using a state-of-the-art depth estimation pipeline, constructing a keypoint sequence over multiple frames from the reference dataset, and comparing these keypoints to the keypoints extracted from a single query image. We evaluate the system on a challenging benchmark dataset and show that it consistently outperforms state-of-the-art techniques. We also develop a range of diagnostic simulation experiments that characterize the contribution of depth-filtered keypoint sequences with respect to key domain parameters including the degree of appearance change and camera motion.

Original languageEnglish
Title of host publication2019 International Conference on Robotics and Automation (ICRA)
EditorsJaydev P. Desai
Place of PublicationDanvers MA USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages4916-4923
Number of pages8
ISBN (Electronic)9781538660263
DOIs
Publication statusPublished - 2019
EventIEEE International Conference on Robotics and Automation 2019 - Montreal, Canada
Duration: 20 May 201924 May 2019

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)1050-4729

Conference

ConferenceIEEE International Conference on Robotics and Automation 2019
Abbreviated titleICRA 2019
CountryCanada
CityMontreal
Period20/05/1924/05/19

Cite this

Garg, S., Babu, M. V., Dharmasiri, T., Hausler, S., Suenderhauf, N., Kumar, S., ... Milford, M. (2019). Look no deeper: recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation. In J. P. Desai (Ed.), 2019 International Conference on Robotics and Automation (ICRA) (pp. 4916-4923). (Proceedings - IEEE International Conference on Robotics and Automation). Danvers MA USA: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICRA.2019.8794178
Garg, Sourav ; Babu, Madhu V. ; Dharmasiri, Thanuja ; Hausler, Stephen ; Suenderhauf, Niko ; Kumar, Swagat ; Drummond, Tom ; Milford, Michael. / Look no deeper : recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation. 2019 International Conference on Robotics and Automation (ICRA). editor / Jaydev P. Desai. Danvers MA USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. pp. 4916-4923 (Proceedings - IEEE International Conference on Robotics and Automation).
@inproceedings{430b0d9cadbe450aa46bab0f9110ed69,
title = "Look no deeper: recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation",
abstract = "Visual place recognition (VPR) - the act of recognizing a familiar visual place - becomes difficult when there is extreme environmental appearance change or viewpoint change. Particularly challenging is the scenario where both phenomena occur simultaneously, such as when returning for the first time along a road at night that was previously traversed during the day in the opposite direction. While such problems can be solved with panoramic sensors, humans solve this problem regularly with limited field-of-view vision and without needing to constantly turn around. In this paper, we present a new depth- and temporal-aware visual place recognition system that solves the opposing viewpoint, extreme appearance-change visual place recognition problem. Our system performs sequence-to-single frame matching by extracting depth-filtered keypoints using a state-of-the-art depth estimation pipeline, constructing a keypoint sequence over multiple frames from the reference dataset, and comparing these keypoints to the keypoints extracted from a single query image. We evaluate the system on a challenging benchmark dataset and show that it consistently outperforms state-of-the-art techniques. We also develop a range of diagnostic simulation experiments that characterize the contribution of depth-filtered keypoint sequences with respect to key domain parameters including the degree of appearance change and camera motion.",
author = "Sourav Garg and Babu, {Madhu V.} and Thanuja Dharmasiri and Stephen Hausler and Niko Suenderhauf and Swagat Kumar and Tom Drummond and Michael Milford",
year = "2019",
doi = "10.1109/ICRA.2019.8794178",
language = "English",
series = "Proceedings - IEEE International Conference on Robotics and Automation",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
pages = "4916--4923",
editor = "Desai, {Jaydev P.}",
booktitle = "2019 International Conference on Robotics and Automation (ICRA)",
address = "United States of America",

}

Garg, S, Babu, MV, Dharmasiri, T, Hausler, S, Suenderhauf, N, Kumar, S, Drummond, T & Milford, M 2019, Look no deeper: recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation. in JP Desai (ed.), 2019 International Conference on Robotics and Automation (ICRA). Proceedings - IEEE International Conference on Robotics and Automation, IEEE, Institute of Electrical and Electronics Engineers, Danvers MA USA, pp. 4916-4923, IEEE International Conference on Robotics and Automation 2019, Montreal, Canada, 20/05/19. https://doi.org/10.1109/ICRA.2019.8794178

Look no deeper : recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation. / Garg, Sourav; Babu, Madhu V.; Dharmasiri, Thanuja; Hausler, Stephen; Suenderhauf, Niko; Kumar, Swagat; Drummond, Tom; Milford, Michael.

2019 International Conference on Robotics and Automation (ICRA). ed. / Jaydev P. Desai. Danvers MA USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. p. 4916-4923 (Proceedings - IEEE International Conference on Robotics and Automation).

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Look no deeper

T2 - recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation

AU - Garg, Sourav

AU - Babu, Madhu V.

AU - Dharmasiri, Thanuja

AU - Hausler, Stephen

AU - Suenderhauf, Niko

AU - Kumar, Swagat

AU - Drummond, Tom

AU - Milford, Michael

PY - 2019

Y1 - 2019

N2 - Visual place recognition (VPR) - the act of recognizing a familiar visual place - becomes difficult when there is extreme environmental appearance change or viewpoint change. Particularly challenging is the scenario where both phenomena occur simultaneously, such as when returning for the first time along a road at night that was previously traversed during the day in the opposite direction. While such problems can be solved with panoramic sensors, humans solve this problem regularly with limited field-of-view vision and without needing to constantly turn around. In this paper, we present a new depth- and temporal-aware visual place recognition system that solves the opposing viewpoint, extreme appearance-change visual place recognition problem. Our system performs sequence-to-single frame matching by extracting depth-filtered keypoints using a state-of-the-art depth estimation pipeline, constructing a keypoint sequence over multiple frames from the reference dataset, and comparing these keypoints to the keypoints extracted from a single query image. We evaluate the system on a challenging benchmark dataset and show that it consistently outperforms state-of-the-art techniques. We also develop a range of diagnostic simulation experiments that characterize the contribution of depth-filtered keypoint sequences with respect to key domain parameters including the degree of appearance change and camera motion.

AB - Visual place recognition (VPR) - the act of recognizing a familiar visual place - becomes difficult when there is extreme environmental appearance change or viewpoint change. Particularly challenging is the scenario where both phenomena occur simultaneously, such as when returning for the first time along a road at night that was previously traversed during the day in the opposite direction. While such problems can be solved with panoramic sensors, humans solve this problem regularly with limited field-of-view vision and without needing to constantly turn around. In this paper, we present a new depth- and temporal-aware visual place recognition system that solves the opposing viewpoint, extreme appearance-change visual place recognition problem. Our system performs sequence-to-single frame matching by extracting depth-filtered keypoints using a state-of-the-art depth estimation pipeline, constructing a keypoint sequence over multiple frames from the reference dataset, and comparing these keypoints to the keypoints extracted from a single query image. We evaluate the system on a challenging benchmark dataset and show that it consistently outperforms state-of-the-art techniques. We also develop a range of diagnostic simulation experiments that characterize the contribution of depth-filtered keypoint sequences with respect to key domain parameters including the degree of appearance change and camera motion.

UR - http://www.scopus.com/inward/record.url?scp=85071473427&partnerID=8YFLogxK

U2 - 10.1109/ICRA.2019.8794178

DO - 10.1109/ICRA.2019.8794178

M3 - Conference Paper

AN - SCOPUS:85071473427

T3 - Proceedings - IEEE International Conference on Robotics and Automation

SP - 4916

EP - 4923

BT - 2019 International Conference on Robotics and Automation (ICRA)

A2 - Desai, Jaydev P.

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Danvers MA USA

ER -

Garg S, Babu MV, Dharmasiri T, Hausler S, Suenderhauf N, Kumar S et al. Look no deeper: recognizing places from opposing viewpoints under varying scene appearance using single-view depth estimation. In Desai JP, editor, 2019 International Conference on Robotics and Automation (ICRA). Danvers MA USA: IEEE, Institute of Electrical and Electronics Engineers. 2019. p. 4916-4923. (Proceedings - IEEE International Conference on Robotics and Automation). https://doi.org/10.1109/ICRA.2019.8794178