Development of Data Fusion Signal Processing Algorithms for LiDAR, RADAR, and Stereo Camera Sensors

Joo Han Lee, Yong Hun Kim, Eung Ju Kim, Bo Sung Ko, Min Ho Lee, Jin Woo Song

Abstract: In recent years, the vehicle market has focused on the development of autonomous vehicles as the most important research topic. While extensive research has been conducted on this matter, the safety issues such as handling unexpected situations during driving have not been completely addressed yet. In the context of ensuring safety for the development of autonomous vehicles, the recognition of objects in front of the vehicle is a highly important research topic. The types of sensors used for the development of recognition algorithms are highly diverse, but among them, LiDAR, RADAR and Stereo cameras can be considered as representative. These three types of sensors measure data through different principles, and as a result, they have complementary advantages and disadvantages in terms of aspects like measurement distance, resolution, and price. In most research papers, rather than using a single type of sensor, a process of fusing two or more sensors is performed to develop recognition algorithms. In this context, there has been a complexity in acquiring sensor information required for developing the desired algorithm, as it necessitated individual signal processing procedures for sensors capable of providing the necessary output information. In this research, an integrated signal processing algorithm is developed to acquire the output information and accuracy of three types of sensors in a single data format. LiDAR, RADAR and Stereo cameras each provide distance, speed, and image data with different field of view characteristics. The data formats output from each sensor also varies, including point cloud and pixel image, among others. LiDAR can output distance data in point cloud format, while RADAR can provide distance and velocity data in point cloud format. Additionally, Stereo cameras can output two-dimensional pixel image data in Red, Green, and Blue (RGB) color. To develop the signal processing algorithm that fuses these into a single data format, Stereo camera based RGB image data is initially chosen as the fundamental base. Then, the data output in point cloud format from LiDAR and RADAR is converted into the image data format, consistent with Stereo camera. Next, a selection process is performed on the transformed LiDAR and RADAR based image data to retain data within the field of view of the Stereo camera. All three types of image data now have a same field of view. One common characteristic among the data from these three sensors is the presence of depth information. Therefore, a fusion process based on filters is employed to combine the depth information from each sensor into a single depth and probability information. Subsequently, signal processing algorithms are applied by mapping the fused depth and probability information to each pixel of the Stereo camera based RGB image data in a one-to-one manner. Through this integrated signal processing algorithm, it is possible to obtain data in the RGB image, Depth and Probability (RGBDP) format, which includes both distance and probability information, from a single integrated image data. By applying the integrated signal processing algorithm developed in this study, it becomes possible to acquire three-dimensional information about the vehicle’s front without the need for separate signal processing procedures for each sensor.
Published in: Proceedings of the 2024 International Technical Meeting of The Institute of Navigation
January 23 - 25, 2024
Hyatt Regency Long Beach
Long Beach, California
Pages: 1064 - 1073
Cite this article: Lee, Joo Han, Kim, Yong Hun, Kim, Eung Ju, Ko, Bo Sung, Lee, Min Ho, Song, Jin Woo, "Development of Data Fusion Signal Processing Algorithms for LiDAR, RADAR, and Stereo Camera Sensors," Proceedings of the 2024 International Technical Meeting of The Institute of Navigation, Long Beach, California, January 2024, pp. 1064-1073.
Full Paper: ION Members/Non-Members: 1 Download Credit
Sign In