Towards the development of standardized methods for comparison, ranking and evaluation of structure alignments

Pairwise alignment of protein structures is a fundamental task in structural bioinformatics. There are numerous computer programs in the public domain that produce alignments for a given pair of protein structures, but the results obtained by the various programs generally differ substantially. Hence, in the application of such programs the question arises which of the alignment programs are the most trustworthy in the sense of overall performance, and which programs provide the best result for a given pair of proteins. The major problem in comparing, evaluating and judging alignment results is that there is no clear notion of the optimality of an alignment. As a consequence, the numeric criteria and scores reported by the individual structure alignment programs are largely incomparable.

Here we report on the development and application of a new approach for the evaluation of structure alignment results. The method uses the translation vector and rotation matrix to generate the superposition of two structures but discards the alignment reported by the individual programs. The optimal alignment is then generated in standardized form based on a suitably implemented dynamic programming algorithm where the length of the alignment is the single most informative parameter. We demonstrate that some of the most popular programs in protein structure research differ considerably in their overall performance. In particular, each of the programs investigated here produced in at least in one case the best and the worst alignment compared with all others. Hence, at the current state of development of structure comparison techniques, it is advisable to use several programs in parallel and to choose the optimal alignment in the way reported here.

The computer software that implement the method described here is freely available at http://melolab.org/stovca.