Integrated access to big data polystores through a knowledge-driven framework

TLDR

Knowledge graphs are widely adopted as a single source of truth, yet most organizations’ big data—time series, images, and unstructured text—cannot be efficiently stored within a graph. This paper introduces SemTK, a framework that lets users access polyglot Big Data stores while presenting all data as if it resided in a knowledge graph. SemTK stores data on the most suitable platform (e.g., Hadoop, graph databases, triple stores) and exposes a unified logical interface, creating a knowledge‑driven veneer across heterogeneous sources. Using four GE industrial use cases, the authors demonstrate that SemTK simplifies construction and querying of polystore knowledge graphs, yielding tangible usability and performance benefits.

Abstract

The recent successes of commercial cognitive and AI applications have cast a spotlight on knowledge graphs and the benefits of consuming structured semantic data. Today, knowledge graphs are ubiquitous to the extent that organizations often view them as a "single source of truth" for all of their data and other digital artifacts. In most organizations, however, Big Data comes in many different forms including time series, images, and unstructured text, which often are not suitable for efficient storage within a knowledge graph. This paper presents the Semantics Toolkit (SemTK), a framework that enables access to polyglot-persistent Big Data stores while giving the appearance that all data is fully captured within a knowledge graph. SemTK allows data to be stored across multiple storage platforms (e.g., Big Data stores such as Hadoop, graph databases, and semantic triple stores) - with the best-suited platform adopted for each data type - while maintaining a single logical interface and point of access, thereby giving users a knowledge-driven veneer across their data. We describe the ease of use and benefits of constructing and querying polystore knowledge graphs with SemTK via four industrial use cases at GE.