Flow-based Programming for Real-time Multi-sensor Data Fusion
Thomas Topp and Thomas Hobiger, Institute of Navigation, University of Stuttgart
Date/Time: Thursday, Sep. 22, 8:57 a.m.
Software design for a multi-sensor data fusion application usually focuses on a particular application or is tailored to certain algorithms. While such a software, if carefully coded, will fulfill the design goals in terms of functionality and performance, it has usually little flexibility in terms of adaptability when the set of sensors changes, when algorithms are replaced or when the overall fusion approach follows another logic. Moreover, such a software implementation becomes limited because also data reading, pre-processing and evaluation of results are usually implemented in a single hard-coded processing chain. Additionally third-party tools are often used to handle one or more of these tasks, which enlarges the overall structure of projects. When the software is intended for real-time use, efficient data-handling and dynamic interaction with a suite of sensors is necessary. As a consequence one faces the risk that such a software becomes difficult to maintain and can only be reused after a major redesign of the code. Thus, it is no surprise that for this reason the same basic algorithms have been written over and over again in every new project in the past. This leads to very time-consuming implementation phases in projects and prevents the actual research or product being available earlier. Furthermore, sensor fusion algorithms are usually written and designed by experts, who need to have excellent programming skills in order to ensure real-time processing capability and correctness of the results.
In contrast, flow-based programming allows more flexibility, makes code reusable and allows even inexperienced users without a programming or navigation background to design applications tailored to their needs. By abstracting functionality, like reading a file or plotting the output, into separate modules, so-called nodes, the implementation becomes more self-contained. Nodes by default cannot communicate with each other but provide input and output pins over which data is sent out or received. This adds an additional layer of abstraction to the programming logic, which goes beyond the typical public/private visibility of classes. To determine the path the data should take, pins can be connected by links in a graphical user interface. However, links can only be established between pins of the same or inherited data types, which makes it easy to configure such a software and prevents hard-to-detect bugs due to data type conversion errors. Inside the nodes there are callbacks, which are triggered when data is received. The typical behavior of a node would be, to process the data and construct a new data package, which can again be sent out from an output pin. That way the data flows from one node to the next and this is the reason why it is called flow-based programming.
Pins define exactly what type of data is transmitted and therefore act as a clean interface between nodes. An advantage of this is, that nodes with pins of the same data type can be exchanged with each other. For example, a file reader node can be exchanged with a sensor node and as a result transforming the program from a post-processing tool into a real-time application can be done with a few clicks. Other parts of the data flow stay unchanged and hence real-time algorithms can be developed with data files, which speeds up development, and then be used with sensor data without changing a single line of code. It also does not matter from what manufacturer a sensor is. One just has to write a node, which handles the device-specific low-level communication to the sensor, and convert it into a high-level data message, which can be processed by algorithm nodes.
Because the flow of data can be visualized in a graphical user interface it becomes easier to understand the algorithms. The need to generate UML diagrams also gets reduced drastically as dependencies are directly visible. In this regard flow-based programming is similar to a model-based design approach but still gives full control over the processing sequence and implementation. A graphical user interface also gives the advantage that parameters can be configured for the node they belong to and not all parameters together in a single file. Parameters can also be changed and applied without the need to restart the software or it can even be done while the algorithm is running, if desired. Furthermore the data processing logic can be separated from the graphical user interface. This allows the software to be configured on a desktop PC and then copied over to a productive system and run there without any performance overhead due to the GUI. That is done without the need of an auto code generation functionality, which gives developers entire control of their algorithms and most importantly makes software directly certifiable.
Another benefit from using a flow-based architecture is, that data can be processed in parallel. Nodes at the end of the flow most of the time do not depend on the nodes and data at the start of the flow. Therefore hard limits for execution times become more unconstrained. However, it is still possible to feed back data from a node further down the flow to a node before by implementing simple wait logics and queues. This can be done on the level of the nodes, which requires researchers to only set the parameters for the logic. That also makes it possible to set requirements for the availability of data when having multiple input pins. Algorithms can only be executed, if enough data messages from each input pin are available. For example when data from different sensors becomes available and they can be fused together in a filter algorithm.
In conclusion, the use of flow-based programming greatly simplifies the use and development of navigation software. On the one hand, users with minimal experience in the field of navigation can use existing nodes inside the graphical user interface, connect them and thus customize the data flow to their needs. This makes such software usable for industry, research and also for teaching. On the other hand, navigation experts can focus more on the algorithms, which they want to implement, as basic functionality can easily be reused. This reduces cycle times and makes the process of development more agile.
Our presentation will cover the basic principles of flow-based programming and demonstrate the major advantages of such an approach with respect to existing solutions. Illustrative examples with our software INSTINCT are given in order to show how easy it is to create sensor fusion solutions from scratch, while maintaining full control about the computational logic and the set of parameters that drive the actual fusion algorithms. Thus, the audience will learn about the clear benefits of flow-based programming for real-time multi-sensor data fusion and how such a solution is made available to the community by means of our software INSTINCT.
For Attendees Technical Program Registration Hotel Travel and Visas COVID-19 Information Smartphone Decimeter Challenge Exhibits Submit Kepler Nomination For Authors and Chairs Abstract Management Author Resource Center Session Chair Resources Student Paper Awards For Exhibitors Exhibitor Resource Center Marketing Resources Other Years Future Meetings Past Meetings