From Relative Annotation to Pairwise Learning for Assessing Echocardiographic Rotational Alignment

To evaluate whether preserving relative expert-derived structure during model training improves the automated assessment of rotational misalignment in apical four-chamber echocardiographic images.

A multi-expert dataset of apical four-chamber images was annotated using multiple-image ranking to derive consensus rotational orderings along a clockwise-to-anti-clockwise continuum. Conventional pointwise regression models were compared with shared-weight pairwise models that received two images and predicted the signed difference between their consensus rotational scores. Performance was evaluated across three independent expert-ranked test sets using pairwise agreement and Spearman correlation. Robustness was further assessed using leave-one-expert-out and cycle-consistency analyses, with saliency-based visualization used to examine the image regions influencing predictions.

Pairwise learning increased the mean pairwise agreement from 74.86% to 84.29% and the mean Spearman correlation from 67.74% to 85.43% compared with pointwise regression. The best pairwise model achieved a pairwise agreement of up to 87.81% and a Spearman correlation of up to 91.23%. The performance remained stable when individual annotators were excluded, and a cycle-consistency analysis demonstrated few cyclic preferences on independent test data. Visual explanation maps indicated model attention to clinically relevant cardiac anatomy.

Training directly on relative expert-derived structure improved agreement with multi-expert consensus compared with conventional pointwise regression. Pairwise learning therefore provides a robust framework for assessing qualitative, continuous echocardiographic image-quality attributes such as rotational alignment.