Towards efficient data exchange and sharing for big-data driven materials science: metadata and data formats

TLDR

Big‑data driven materials research demands widespread data sharing, which requires standardization of units, baselines, and file formats, and can be achieved by either converters or open libraries. The paper proposes two complementary strategies—converters for heterogeneous codes and standardized open libraries—to enable code‑independent data exchange. Converters translate code‑specific data into a standardized, code‑independent format, while open libraries provide a common framework and hierarchical metadata for electronic‑structure calculations. The Electronic Structure Library and NOMAD Laboratory have agreed on the appropriate format and conventions.

Abstract

Abstract With big-data driven materials research, the new paradigm of materials science, sharing and wide accessibility of data are becoming crucial aspects. Obviously, a prerequisite for data exchange and big-data analytics is standardization, which means using consistent and unique conventions for, e.g., units, zero base lines, and file formats. There are two main strategies to achieve this goal. One accepts the heterogeneous nature of the community, which comprises scientists from physics, chemistry, bio-physics, and materials science, by complying with the diverse ecosystem of computer codes and thus develops “converters” for the input and output files of all important codes. These converters then translate the data of each code into a standardized, code-independent format. The other strategy is to provide standardized open libraries that code developers can adopt for shaping their inputs, outputs, and restart files, directly into the same code-independent format. In this perspective paper, we present both strategies and argue that they can and should be regarded as complementary, if not even synergetic. The represented appropriate format and conventions were agreed upon by two teams, the Electronic Structure Library (ESL) of the European Center for Atomic and Molecular Computations (CECAM) and the NOvel MAterials Discovery (NOMAD) Laboratory, a European Centre of Excellence (CoE). A key element of this work is the definition of hierarchical metadata describing state-of-the-art electronic-structure calculations.