On-line data validation in distributed data fusion

Jürgo Preden, James Llinas, Galina L. Rogova, Raido Pahtma, L. Môtus

Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2013 · 17 citations · 3 references

Concepts

TL;DR

Data acquisition and fusion systems are increasingly complex systems of systems whose dynamic configurations and lack of synchronous operation require ensuring temporal and spatial data correctness and quality dimensions such as accuracy, confidence, relevance, credibility, and reliability. This paper proposes mediated interactions to guarantee computational correctness in distributed systems. Each computing entity hosts a mediator that performs online validation of data, both before sending and upon receipt, using metadata to enforce receiver‑side validity constraints. The mediator handles unpredictable runtime temporal uncertainties like channel delay, clock jitter, and processing delays, proving effective in multitasking scenarios and variable parallel data streams, as demonstrated by the presented architecture and simulation case study.

Abstract

Data acquisition and data fusion systems are becoming increasingly complex, being in fact systems of systems, where every component may be a system with varying levels of autonomy by themselves. Possible changes in system configuration by entities joining or being removed from the system make the system complex. As synchronous operation cannot be expected in such a system configuration, the temporal and spatial correctness of data must be achieved via other means. This paper presents the concept of mediated interactions as a method for ensuring correctness of computation in a distributed system. The mediator associated with each computing entity is responsible for online checking of the data both before it is sent out at the sender side and before it is received at the receiver side, ensuring that only data satisfying the validity constraints of the receiver-side data processing algorithm is used in computation. This assumes that each data item is augmented with metadata, which enables online data validation. The validity and quality dimensions in use depend on the system requirements defined by a specific problem and situational context; they may be temporal, spatial and involve various data quality dimensions, such as accuracy, confidence, relevance, credibility, and reliability. Among other capabilities, the mediator is able to cope with the unknowns in the temporal dimension that occur at runtime and are not predictable, such as channel delay, jitter of clocks and processing delays. This capability becomes an especially relevant factor in multi-tasking systems and in configurations in which a computing entity may have to process a variable number of parallel streams of data. Both the architecture and a simulation case study of a distributed data fusion scenario are presented in the paper.

References

3