Dark Data as the New Challenge for Big Data Science and the Introduction of the Scientific Data Officer

Björn Schembera, Juan M. Durán

Philosophy & Technology · 2019 · 66 citations · 20 references

DOIFull text

Open access

Concepts

TL;DR

Big‑data research typically focuses on data that researchers can access, leaving behind dark data stored on servers that remains largely unexamined. This article introduces the concept of dark data in HPC environments and proposes the scientific data officer (SDO) position to manage it. The authors analyze statistics from the High‑Performance Computing Center Stuttgart, define the SDO’s technical and ethical responsibilities, and distinguish the role from other HPC positions. They find that the SDO’s technical duties characterize the role, while its ethical responsibilities highlight control and authority concerns and suggest solutions.

Abstract

Many studies in big data focus on the uses of data available to researchers, leaving without treatment data that is on the servers but of which researchers are unaware. We call this dark data, and in this article, we present and discuss it in the context of high-performance computing (HPC) facilities. To this end, we provide statistics of a major HPC facility in Europe, the High-Performance Computing Center Stuttgart (HLRS). We also propose a new position tailor-made for coping with dark data and general data management. We call it the scientific data officer (SDO) and we distinguish it from other standard positions in HPC facilities such as chief data officers, system administrators, and security officers. In order to understand the role of the SDO in HPC facilities, we discuss two kinds of responsibilities, namely, technical responsibilities and ethical responsibilities. While the former are intended to characterize the position, the latter raise concerns—and proposes solutions—to the control and authority that the SDO would acquire.

References

20