On effective and optical resolutions of diffraction data sets

Abstract

In macromolecular X-ray crystallography, diffraction data sets are traditionally characterized by the highest resolutiondhighof the reflections that they contain. This measure is sensitive to individual reflections and does not refer to the eventual data incompleteness and anisotropy; it therefore does not describe the data well. A physically relevant and robust measure that provides a universal way to define the `actual' effective resolutiondeffof a data set is introduced. This measure is based on the accurate calculation of the minimum distance between two immobile point scatterers resolved as separate peaks in the Fourier map calculated with a given set of reflections. This measure is applicable to any data set, whether complete or incomplete. It also allows characterizion of the anisotropy of diffraction data sets in whichdeffstrongly depends on the direction. Describing mathematical objects, the effective resolutiondeffcharacterizes the `geometry' of the set of measured reflections and is irrelevant to the diffraction intensities. At the same time, the diffraction intensities reflect the composition of the structure from physical entities: the atoms. The minimum distance for the atoms typical of a given structure is a measure that is different from and complementary todeff; it is also a characteristic that is complementary to conventional measures of the data-set quality. Following the previously introduced terms, this value is called the optical resolution,dopt. The optical resolution as defined here describes the separation of the atomic images in the `ideal' crystallographic Fourier map that would be calculated if the exact phases were known. The effective and optical resolution, as formally introduced in this work, are of general interest, giving a common `ruler' for all kinds of crystallographic diffraction data sets.