Quantifying Navigation Safety of Autonomous Passenger Vehicles (APVs)

Mathieu Joerger, Matthew Spenko

Abstract:	This work describes the major research challenges encountered in developing navigation safety standards for self-driving cars. The key question is: “how can we prove that an autonomous car will not injure a human because of localization errors?”. In response, our approach aims at evaluating and guaranteeing integrity, which is a measure of trust in a sensor’s information. For decades, integrity has been the primary navigation performance metric for life-critical, autonomous aviation applications that have a proven safety record. In this work, we investigate how to extend the concept of integrity to the more challenging case of Autonomous Passenger Vehicles (APVs). The results will be a quantifiable, sensor- and platform-independent navigation safety standard that will contribute to the growth and maturation of APV navigation technology. APVs may help prevent car accidents that cause over 30,000 deaths/year, cost approximately $230 billion/year in medical and work loss costs, and are caused by humans 90% of the time. Multiple efforts are currently undertaken to prove APV safety. Based on publicly available information, it appears that safety-evaluation techniques, such as those employed by Google and Tesla, use a brute force, experimental-only approach where the best outcome is that an APV is “safe” because it has driven a set number of miles without an accident. This is problematic because the approach would require billions of driven miles to just match current manned vehicle accident rates, not to mention rates similar to the aviation industry. This is orders of magnitude larger than any current effort. Furthermore, during that testing period any navigation errors, sensor changes, or algorithm modifications would necessitate the experiment to restart. Last, experiments would need to represent all US roads under all weather conditions. In contrast, we are developing a method that relies on limited experimental data coupled with analytical tools to prove that the probability of an APV’s future pose lies within acceptable limits (similar to approaches used in aviation applications). In turn, the approach will provide timely warnings when the APV anticipates a loss of integrity in order to execute an appropriate emergency maneuver. Establishing APV integrity evaluation methods holds new and significant challenges compared to prior work in aviation, which relies heavily on GPS. In contrast, APVs require multiple and varied sensors to compensate for GPS signal blockages caused by buildings and trees. Thus, we must model statistical bounds on measurement errors for non-GPS sensors. An example method to upper-bound the integrity risk of laser scanner data association is given in [1]. We must also integrate different sensor types, and develop new methods to evaluate the integrity of multi-sensor systems. Furthermore, APVs experience continuous risk in a constantly changing environment, whereas aviation risk only peaks during landing. Therefore, we need to continuously predict integrity in a dynamic APV environment. If the resulting challenges are overcome, one could quantify and prove the performance of an APV’s navigation system. In addition, one could allocate integrity risk requirements to individual system components and thus set safety-based design guidelines for each sensor. Finally, as APV technology progresses from driver’s aids such as active brake assist to full autonomous driving, this research is relevant now and will remain essential throughout the evolution of APV technology. [1] Joerger, Mathieu, Jamoom, Michael, Spenko, Matthew, Pervan, Boris, "Integrity of Laser-Based Feature Extraction and Data Association," Proceedings of IEEE/ION PLANS 2016, Savannah, GA, April 2016, pp. 557-571.
Published in:	Proceedings of the 29th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2016) September 12 - 16, 2016 Oregon Convention Center Portland, Oregon
Pages:	1531 - 1557
Cite this article:	Joerger, Mathieu, Spenko, Matthew, "Quantifying Navigation Safety of Autonomous Passenger Vehicles (APVs)," Proceedings of the 29th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2016), Portland, Oregon, September 2016, pp. 1531-1557. https://doi.org/10.33012/2016.14551
Full Paper:	ION Members/Non-Members: 1 Download Credit Sign In