Reinforcement Learning Framework for Robust Navigation in GNSS Receivers

David Contreras Franco, Iñigo Cortés, Georgios Kontes, Tobias Feigl, Christopher Mutschler, and Alexander Rügamer

Peer Reviewed

Abstract:	Robust synchronization of the code phase, carrier Doppler, and carrier phase is essential in global navigation satellite system (GNSS) receivers to achieve continuous and reliable position, velocity, and time (PVT) solutions. Traditional fixed-configuration tracking loops are sub-optimal for time-varying scenarios. Recent research introduced the loop-bandwidth control algorithm (LBCA), an adaptive technique that updates the loop bandwidth based on the weighted difference between the mean and standard deviation of the discriminator’s output. Extensive testing has confirmed the LBCA’s superior performance compared to state-of-the-art adaptive tracking techniques while maintaining low complexity. However, the primary limitation lies in the need for precise tuning of features and weighting functions, which are traditionally hand-crafted and may neglect other effects such as transient dynamics or Allan variance from the oscillator. This paper presents the reinforcement learning bandwidth adaptive control (RLBAC), a reinforcement learning (RL) framework to optimize the adaptive control algorithm by searching for optimal weighting functions based on selected features. The environment in this framework represents the tracking channels of a GNSS receiver, while the agent includes the LBCA. The REINFORCE algorithm, a policy search/gradient algorithm, is included to optimize the policy function parameters (i.e., the weights of the LBCA). Two setups were implemented: first, an RL framework in Python to train the LBCA using synthetic signals; second, evaluation within the GOOSE©platform, a GNSS receiver with an open software interface. Results demonstrate that the LBCA can be the policy function of a reinforcement learning framework and that it is trainable, allowing for the search of optimal weighting functions. This proof-of-concept highlights the potential for RL frameworks to enhance adaptive control in GNSS tracking.
Published in:	Proceedings of the 37th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2024) September 16 - 20, 2024 Hilton Baltimore Inner Harbor Baltimore, Maryland
Pages:	2392 - 2408
Cite this article:	Franco, David Contreras, Cortés, Iñigo, Kontes, Georgios, Feigl, Tobias, Mutschler, Christopher, Rügamer, Alexander, "Reinforcement Learning Framework for Robust Navigation in GNSS Receivers," Proceedings of the 37th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2024), Baltimore, Maryland, September 2024, pp. 2392-2408. https://doi.org/10.33012/2024.19853
Full Paper:	ION Members/Non-Members: 1 Download Credit Sign In