ePrints Soton (University of Southampton) · 2018 · 17 citations · 8 references
ComputingComputer ScienceHigh-performance ComputingData EngineeringInstrumentationScientific ComputingAccelerator IntegrationData Acquisition SystemsDistributed Data ProcessingExperimentation PlatformsHardware AccelerationReal-time Data AcquisitionData-intensive ComputingHardware InstrumentationInstrumentation Control SystemsInstrumentation SoftwareOffline Data ProcessingPerformance RoutinesReal-time Data AnalysisReal-time Data Processing
We describe the data analysis structure that is integrated into the Karabo framework [1] to support scientific experiments and data analysis at European XFEL GmbH. The photon science experiments have a range of data analysis requirements, including online (i.e. near real-time during the actual measurement) and offline data analysis. The Karabo data analysis framework supports execution of automatic data analysis for routine tasks, supports complex experiment protocols including data analysis feedback integration to instrument control, and supports integration of external applications. The online data analysis is carried out using distributed and accelerator hardware (such as GPUs) where required to balance load and achieve near real-time data analysis throughput. Analysis routines provided by Karabo are implemented in C++ and Python, and make use of established scientific libraries. The XFEL control and analysis software team collaborates with users to integrate experiment specific analysis codes, protocols and requirements into this framework, and to make it available for the experiments and subsequent offline data analysis.
8
An introduction to Docker for reproducible research
Carl Boettiger · ACM SIGOPS Operating Systems Review · 2015 · 1.1K citations · Full text
Development of the LPD, a high dynamic range pixel detector for the European XFEL
Mathew Hart, C. Angelsen, S. Burge et al. · 2012 · 55 citations
Engineering, Computer Engineering, Electrical Engineering +17