A Hybrid Deep Learning Approach for Robust Multi-Sensor GNSS/INS/VO Fusion in Urban Canyons

Patrick Geragersian, Ivan Petrunin, Weisi Guo, Raphael Grech

Abstract: This paper addresses the significant challenges of robust autonomous navigation in Unmanned Aerial Vehicles (UAVs) within densely populated environments. The focus is on enhancing the performance of Position, Navigation, and Timing (PNT), as specified by the International Civil Aviation Organization, in terms of accuracy, integrity, continuity, and availability. The novel contribution introduces a Robust Multi-Sensor Fusion Architecture (RMSFA) that utilizes a Bayesian-LSTM machine learning algorithm, fusing GNSS, INS, and monocular odometry. Unlike existing solutions that rely on sensor redundancies or methods such as Receiver Autonomous Integrity Monitoring (RAIM), which have limitations in performance, or adaptability to erroneous signals, the proposed system offers improvements in both positioning accuracy and integrity. Furthermore, GNSS data is preprocessed to remove NoneLine-of-Sight data (NLOS) to improve positioning accuracy. Additionally, INS data errors are corrected using a GRU-based error correction architecture to improve INS positioning and reduce drifting. The addition of these post-processing steps reduced the 95th percentile horizontal error by 97.4% and 71.5% respectively. A CNN-LSTM architecture is used to obtain a Visual Odometer (VO) from the camera sensor. The Bayesian-LSTM architecture fusion performance was then compared to a GNSS/IMU/VO EKF-GRU architecture. The comparison showed a 95th percentile error improvement in the horizontal direction of 30.1% for the BayesianLSTM. The architecture was tested in a realistic simulated environment utilizing Unreal Engine and AirSim for UAV simulation, Spirent GNSS7000 simulator for Hardware-in-the-Loop (HIL) simulation, and OKTAL-SE Sim3D to mimic the effects of multipath on GNSS signals. Overall, this work represents a step toward improving the safety and effectiveness of drone navigation by providing a more robust navigation system suitable for safety-critical situations, without the stated disadvantages in previously mentioned literatures.
Published in: Proceedings of the 36th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2023)
September 11 - 15, 2023
Hyatt Regency Denver
Denver, Colorado
Pages: 2624 - 2643
Cite this article: Geragersian, Patrick, Petrunin, Ivan, Guo, Weisi, Grech, Raphael, "A Hybrid Deep Learning Approach for Robust Multi-Sensor GNSS/INS/VO Fusion in Urban Canyons," Proceedings of the 36th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2023), Denver, Colorado, September 2023, pp. 2624-2643. https://doi.org/10.33012/2023.19271
Full Paper: ION Members/Non-Members: 1 Download Credit
Sign In