Localising in complex scenes using Balanced Adversarial Adaptation

Gil Avraham, Yan Zuo, Tom Drummond

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Domain adaptation and generative modelling have collectively mitigated the expensive nature of data collection and labelling by leveraging the rich abundance of accurate, labelled data in simulation environments. In this work, we study the performance gap that exists between representations optimised for localisation on simulation environments and the application of such representations in a real-world setting. Our method exploits the shared geometric similarities between simulation and real-world environments whilst maintaining invariance towards visual discrepancies. This is achieved by optimising a representation extractor to project both simulated and real representations into a shared representation space. Our method uses a symmetrical adversarial approach which encourages the representation extractor to conceal the domain that features are extracted from and simultaneously preserves robust attributes between source and target domains that are beneficial for localisation. We evaluate our method by adapting representations optimised for indoor Habitat simulated environments (Matterport3D and Replica) to a real-world indoor environment (Active Vision Dataset), showing that it compares favourably against fully-supervised approaches.

Original languageEnglish
Title of host publicationProceedings - 2020 International Conference on 3D Vision, 3DV 2020
EditorsTakeshi Masuda
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages11
ISBN (Electronic)9781728181288
ISBN (Print)9781728181295
Publication statusPublished - 2020
EventInternational Conference on 3D Vision 2020 - Online, Fukuoka, Japan
Duration: 25 Nov 202028 Nov 2020
Conference number: 8th
https://ieeexplore-ieee-org.ezproxy.lib.monash.edu.au/xpl/conhome/9320095/proceeding (Proceedings)
http://3dv2020.dgcv.nii.ac.jp (Website)

Publication series

NameProceedings - 2020 International Conference on 3D Vision, 3DV 2020
PublisherThe Institute of Electrical and Electronics Engineers, Inc.
ISSN (Print)2378-3826
ISSN (Electronic)2475-7888


ConferenceInternational Conference on 3D Vision 2020
Abbreviated title3DV 2020
Internet address


  • Domain Adaptation
  • Generative Adversarial Networks
  • Localisation
  • Sim2Real

Cite this