Previous Abstract Return to Session A10

Session A10: Operational System Demonstrations 2

CBIP: Contextual Behavioral Intelligence Platform for Localization and Navigation in GPS Denied Environment.
Michael Park, Devu M. Shila, Unknot.id Inc.; Frank M Tucker, U.S. Army CCDC SC
Location: Ballroom A
Date/Time: Thursday, Jun. 15, 1:45 p.m.

By far the most pressing and consequential need for an infrastructure-less solution to the problem of positioning, navigation, and timing (PNT), is that of the warfighter on the battlefield, where external navigational infrastructures (e.g., Wi-Fi, Bluetooth, or Cellular) are not only guaranteed to be unreliable and unpredictable, but also vulnerable to manipulation and jamming from adversarial agents. Unfortunately, there exist strong physical limitations to the development of a totally infrastructure-less solution to PNT. This goal should thus be viewed as an unattainable ideal, but also one that is asymptotically approachable. In short, infrastructure-less PNT of humans or any assets within areas of dense infrastructure (e.g., indoor spaces with one or multiple floors), triple canopy or urban canyons, where GPS is unreliable or intentionally denied, is a challenging problem for which there is currently no de facto solution. IMU (Inertial Measurement Unit)-based navigation solutions, on the other hand, are particularly appealing as infrastructure-less navigation solutions. Despite the multi-faceted advantages such as cost-effectiveness, infrastructure-less, highly energy-efficient, and far less environment dependent, accurate pose (position and orientation) estimation using IMUs are notoriously hard due to drifting with unbounded error caused by sensor noises, biases, and contexts (refers to human motion dynamics such as walking, running etc., device placements such as in chest or in hand, and user-specific attributes). ZUPT-aided (Zero-Velocity Update) inertial navigation and step-based PDR (Pedestrian Dead Reckoning) are techniques proposed to amortize the cost of unbounded drifting in IMU’s; but they are limited by assumptions on motion dynamics and sensor placements that in turn prevent its widespread adoption in complex real-life navigation scenarios (e.g., jumping, crawling, rapid turns etc.). Existing Kalman filtering based pose estimation also faces severe issues with computing the linear acceleration caused by sensor motion while simultaneously estimating sensor noises, biases, and contexts. Lately the classical field of IMU-based (or inertial) navigation with low-cost noisy IMU sensors, as the only source of information, has begun to receive attention from novel deep neural network algorithms. However, most of these solutions has remained confined to the academic literature and has not yet been tested for in-the-wild scenarios, with experiments that are invariantly restricted to carefully curated data streams (aka controlled datasets) in simple environments such as malls and office buildings, which in turns limits its ability to generalize to new environments with unseen maneuvers or unseen users; where inertial data in the sensor frame is significantly different from what is observed during the training stages. Moreover, temporal deep neural network architectures explored in these efforts such as Long Short-Term Memory (LSTM), Recurrent Neural Networks (RNN) and Temporal Convolutional Neural Networks (T-CNN) are resource-intensive (e.g., not optimized to run on edge devices) and fails to capture long-term temporal dependencies and spatial correlations between the IMU sensors (3D accelerometer and 3D gyroscope) in multivariate time-series data streams. A careful evaluation of these existing solution’s practical applicability to in-the-wild conditions should thus be a high priority, particularly with regards to continuous, long range (>1hr) memory-efficient navigation, which may include complex motions on multi-level, large infrastructures containing obstacles and narrow pathways. To address this objective, Unknot.id Inc., with the funding from the US Army has developed a game changing contextual, deep learning-based localization and navigation technology called “Contextual Behavioral Intelligence Platform” (CBIP), that exploits memory-efficient, large, spatiotemporal language models on cheap inertial navigation sensors embedded in commercial off the shelf (COTS) devices such as smartphones, wearables etc., to localize and track humans within dense infrastructures with one or more floors. Despite CBIP’s dramatic performance over traditional inertial navigation systems based on Kalman Filters as well as existing neural network architectures such as LSTMs and T-CNN by upwards of ~40% in positional accuracy, when tested and validated on benchmarked and realistic datasets, all IMU-based navigation systems suffer from irreducible cumulative sensor drift that has traditionally limited their adoption for long-range navigation (>1hr). For this reason, while existing indoor PNT solutions currently depend on bespoke hardware infrastructures such as Bluetooth, UWB, Wi-Fi, which cannot be assumed to exist in modern battlefield and soldier training environments, CBIP operates free of any hardware infrastructure by leveraging an innovative graph inspired, deep learning based context-dependent continuous calibration system, which relies only on minimal prior knowledge about the environmental constraints that can be obtained from floor plans (indoors) or topographical maps (e.g., One World Terrain datasets). With strategic partnership with one of the world’s biggest casinos and with ESRI - a global leader in geo location intelligence company, Unknot.id performed a live demonstration of CBIP technology, within Casino’s multi-level, large infrastructure with narrow pathways, stairways, and underground tunnels, where a group of blue/red security personnel were localized using IMU sensor data from smartphones and floor maps, with less than 0.2-meter percentage of distance travelled (PDT) error in near real-time (i.e., predict every 5 to 6 seconds using publicly available communication infrastructure). As part of the operational systems demonstration, the Unknot.id team will walk through the CBIP technology elements and provide a video demonstration of the technology in a challenging, multi-level GPS-denied environment. We will also perform a live demonstration of the technology powered by Android smartphones, AWS and ESRI ArcGIS, by merely relying on sensors of the smartphone and the floor map of the venue. Contrary to current Position, Navigation and Timing (PNT) technologies, the success of CBIP will enable long-range (>1 hr.), highly precise (<0.2% error), 100% localization and navigation availability for challenging, lengthy missions, encompassing complex human maneuvers in all adversarial and non-adversarial GPS-denied environments. It will support an array of personnel and asset (e.g., ground vehicles to drones and weapons) tracking use cases using commercial grade low-cost sensor platforms within DoD, while achieving large gains in accuracy, cost, power consumption, and computational resources. The performance metrics of CBIP resolve critical capability gaps for both Live training and for dismounted Soldier navigation in GPS denied environments.

Previous Abstract Return to Session A10