Data processing and analysis with the<i>autoPROC</i>toolbox

TLDR

High‑throughput diffraction experiments generate many images that overwhelm users and challenge beamline operations, requiring diagnosis of crystal‑handling and collection‑strategy issues to distinguish controllable from uncontrollable problems. The study aims to illustrate common early‑processing problems that arise when converting raw diffraction images into a processed data set and electron‑density model. autoPROC integrates third‑party processing tools with new utilities and an automated workflow script to guide users through offline processing, especially for multi‑sweep data sets collected on multi‑axis goniostats.

Abstract

A typical diffraction experiment will generate many images and data sets from different crystals in a very short time. This creates a challenge for the high-throughput operation of modern synchrotron beamlines as well as for the subsequent data processing. Novice users in particular may feel overwhelmed by the tables, plots and numbers that the different data-processing programs and software packages present to them. Here, some of the more common problems that a user has to deal with when processing a set of images that will finally make up a processed data set are shown, concentrating on difficulties that may often show up during the first steps along the path of turning the experiment (i.e. data collection) into a model (i.e. interpreted electron density). Difficulties such as unexpected crystal forms, issues in crystal handling and suboptimal choices of data-collection strategies can often be dealt with, or at least diagnosed, by analysing specific data characteristics during processing. In the end, one wants to distinguish problems over which one has no immediate control once the experiment is finished from problems that can be remedied a posteriori. A new software package, autoPROC, is also presented that combines third-party processing programs with new tools and an automated workflow script that is intended to provide users with both guidance and insight into the offline processing of data affected by the difficulties mentioned above, with particular emphasis on the automated treatment of multi-sweep data sets collected on multi-axis goniostats.