Apache IoTDB: A Time Series Database for IoT Applications

TLDR

Industrial IoT deployments generate billions of data points from thousands of devices, demanding time‑series management that supports evolving schemas, periodic collection, strong correlations, delayed arrivals, and high‑concurrency ingestion—needs inadequately met by existing solutions. This paper introduces Apache IoTDB, a time‑series database management system designed to address those industrial IoT requirements. Apache IoTDB employs a native TsFile format with specialized encoding and an engine that efficiently handles delayed data arrivals and query processing. It achieves 10 million inserts per second and processes 1‑day selections of 0.1 million points and 3‑year aggregations of 10 million points in 100 ms, outperforming InfluxDB, TimescaleDB, KairosDB, Parquet, and ORC on real‑world workloads.

Abstract

A typical industrial scenario encounters thousands of devices with millions of sensors, consistently generating billions of data points. It poses new requirements of time series data management, not well addressed in existing solutions, including (1) device-defined ever-evolving schema, (2) mostly periodical data collection, (3) strongly correlated series, (4) variously delayed data arrival, and (5) highly concurrent data ingestion. In this paper, we present a time series database management system, Apache IoTDB. It consists of (i) a time series native file format, TsFile, with specially designed data encoding, and (ii) an IoTDB engine for efficiently handling delayed data arrivals and processing queries. The system achieves a throughput of 10 million inserted values per second. Queries such as 1-day data selection of 0.1 million points and 3-year data aggregation over 10 million points can be processed in 100 ms. Comparisons with InfluxDB, TimescaleDB, KairosDB, Parquet and ORC over real world data loads demonstrate the superiority of IoTDB and TsFile.