A likelihood-based search for the macromolecular position in the crystalline unit cell

The goal of this work is to analyse whether the generalized likelihood criterion can be used to find the best spherical envelope of a macromolecule in a unit cell. A family of spherical envelopes is ranged in accordance with their likelihood values calculated by means of a Monte Carlo-type computer procedure. Two kinds of envelope families were tested. The first one was composed of spherical envelopes of fixed radius but different positions in the unit cell. In the second case, the sphere radii were linked to their centre position so that the total volume occupied by all symmetry-related spheres was roughly equal to the total volume occupied by the real molecule. The experiments showed that when using the first type of envelope the level of the signal for the right solution is higher than the one obtained with the straightforward R-factor-based single-Gaussian-atom search, but spurious maxima (usually placed on the symmetry axes) may still exist. The use of the second type of envelope family reduces the level of the spurious maxima.