Data Reduction and Error Analysis for Accurate Single Crystal Diffraction Intensities

Abstract Principles and procedures of diffraction data processing are described. Reflection integration limits are obtained from a least squares analysis of peak profile widths, based on the principles of convolution synthesis of peak profiles. An approximate, empirical thermal diffuse scattering correction is obtained from a least squares analysis of thermal diffuse scattering (TDS) intensity estimates, based on fitting intersecting straight lines to the background profile near the peak limits. Time-dependent scaling according to standard reference reflection intensities is based on least squares fitted scaling polynomials and may be weighted by anisotropic, intensity-dependent or scattering-angle-dependent factors. Inter-set scaling of data subsets and averaging of replicate and equivalent measurements, which includes several criteria for detecting and rejecting outlier measurements, provide a basis for a bivariate analysis of variance on F2 0 and (sin θ)/λ. Error analysis at each stage of the data processing leading up to the analysis of variance includes propagation of estimated experimental errors, as well as estimates of the statistical errors and of their correlations for the various data processing parameters that are introduced.