MilliEgo: single-chip mmWave radar aided egomotion estimation via deep sensor fusion

Chris Xiaoxuan Lu, Muhamad Risqi U. Saputra, Peijun Zhao, Yasin Almalioglu, Pedro P.B. De Gusmao, Changhao Chen, Ke Sun, Niki Trigoni, Andrew Markham

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

61 Citations (Scopus)


Robust and accurate trajectory estimation of mobile agents such as people and robots is a key requirement for providing spatial awareness for emerging capabilities such as augmented reality or autonomous interaction. Although currently dominated by optical techniques e.g., visual-inertial odometry these suffer from challenges with scene illumination or featureless surfaces. As an alternative, we propose milliEgo, a novel deep-learning approach to robust egomotion estimation which exploits the capabilities of low-cost mm Wave radar. Although mmWave radar has a fundamental advantage over monocular cameras of being metric i.e., providing absolute scale or depth, current single chip solutions have limited and sparse imaging resolution, making existing point-cloud registration techniques brittle. We propose a new architecture that is optimized for solving this challenging pose transformation problem. Secondly, to robustly fuse mmWave pose estimates with additional sensors, e.g. inertial or visual sensors we introduce a mixed attention approach to deep fusion. Through extensive experiments, we demonstrate our proposed system is able to achieve 1.3% 3D error drift and generalizes well to unseen environments. We also show that the neural architecture can be made highly efficient and suitable for real-time embedded applications.

Original languageEnglish
Title of host publicationProceedings of the 18th ACM Conference on Embedded Networked Sensor Systems
EditorsShunsuke Saruwatari
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages14
ISBN (Electronic)9781450375900
Publication statusPublished - 2020
Externally publishedYes
EventACM Conference on Embedded Networked Sensor Systems 2020 - Online, Japan
Duration: 16 Nov 202019 Nov 2020
Conference number: 18th (Proceedings)

Publication series

NameSenSys 2020 - Proceedings of the 2020 18th ACM Conference on Embedded Networked Sensor Systems


ConferenceACM Conference on Embedded Networked Sensor Systems 2020
Abbreviated titleSenSys 2020
Internet address


  • egomotion estimation
  • indoor localization
  • millimeter wave radar

Cite this