Graph-based thermal-inertial SLAM with probabilistic neural networks

Muhamad Risqi U. Saputra, Chris Xiaoxuan Lu, Pedro Porto B. Porto Buarque De Gusmao, Bing Wang, Andrew Markham, Niki Trigoni

Research output: Contribution to journalArticleResearchpeer-review

13 Citations (Scopus)

Abstract

Simultaneous localization and mapping (SLAM) system typically employs vision-based sensors to observe the surrounding environment. However, the performance of such systems highly depends on the ambient illumination conditions. In scenarios with adverse visibility or in the presence of airborne particulates (e.g., smoke, dust, etc.), alternative modalities such as those based on thermal imaging and inertial sensors are more promising. In this article, we propose the first complete thermal-inertial SLAM system that combines neural abstraction in the SLAM front end with robust pose-graph optimization in the SLAM back end. We model the sensor abstraction in the front end by employing probabilistic deep learning parameterized by mixture density networks (MDNs). Our key strategies to successfully model this encoding from thermal imagery are the usage of normalized 14-b radiometric data, the incorporation of hallucinated visual (RGB) features, and the inclusion of feature selection to estimate the MDN parameters. To enable a full SLAM system, we also design an efficient global image descriptor that is able to detect loop closures from thermal embedding vectors. We performed extensive experiments and analysis using three datasets, namely self-collected ground robot and hand-held data taken in indoor environment, and one public dataset (SubT-Tunnel) collected in underground tunnel. Finally, we demonstrate that an accurate thermal-inertial SLAM system can be realized in conditions of both benign and adverse visibility.

Original languageEnglish
Pages (from-to)1875-1893
Number of pages19
JournalIEEE Transactions on Robotics
Volume38
Issue number3
DOIs
Publication statusPublished - 1 Jun 2022

Keywords

  • Loop closure detection
  • pose-graph optimization
  • probabilistic deep neural networks (DNNs)
  • thermal inertial simultaneous localization and mapping (SLAM)

Cite this