Big Data Analytics for Earth Sciences: the EarthServer approach

TLDR

Big Data Analytics has emerged with advanced e‑infrastructures, offering Earth and Environmental sciences the ability to process vast Earth Observation datasets, though challenges arise from geospatial relevance, heterogeneous data models, and complex processing requirements. The study aims to develop tailored techniques and tools for Big Earth Data Analytics. EarthServer provides a coverage‑type analytics engine built on a high‑performance array database, adopting OGC WCS and WCPS standards and offering a holistic stack from query languages to mobile visualization. Its effectiveness is shown by lighthouse applications across marine, geology, atmospheric, planetary, and cryospheric domains.

Abstract

Big Data Analytics is an emerging field since massive storage and computing capabilities have been made available by advanced e-infrastructures. Earth and Environmental sciences are likely to benefit from Big Data Analytics techniques supporting the processing of the large number of Earth Observation datasets currently acquired and generated through observations and simulations. However, Earth Science data and applications present specificities in terms of relevance of the geospatial information, wide heterogeneity of data models and formats, and complexity of processing. Therefore, Big Earth Data Analytics requires specifically tailored techniques and tools. The EarthServer Big Earth Data Analytics engine offers a solution for coverage-type datasets, built around a high performance array database technology, and the adoption and enhancement of standards for service interaction (OGC WCS and WCPS). The EarthServer solution, led by the collection of requirements from scientific communities and international initiatives, provides a holistic approach that ranges from query languages and scalability up to mobile access and visualization. The result is demonstrated and validated through the development of lighthouse applications in the Marine, Geology, Atmospheric, Planetary and Cryospheric science domains.