Interobserver variability of corresponding anatomical landmark placement in pelvic CT and MRI scans

Anatomical landmarks are often used to assess the quality of a deformable image registration (DIR) between two scans. However, such landmarks are manually placed on both scans, which is prone to observer variability. We analyzed the interobserver variability of the placement of corresponding landmarks on pelvic scans, to provide context for DIR validation studies that use landmarks as a reference.

Pelvic CT and MRI scans of nine cervical cancer patients were distributed to 17 observers. Three annotation settings were considered, each including scan pairs of five patients: CT-CT (13 observers), MRI-CT (eight observers), and MRI-MRI (eight observers). The observer group consisted of 15 RTTs professionally trained in working with scans of the given modality, and two fourth-year Ph.D. students in the domain. During annotation, observers received the same reference scan of each patient with 23 anatomically relevant, pre-annotated landmarks, and were asked to annotate the corresponding location of each reference landmark on a second scan of the same patient. To quantify the interobserver variability between different landmark placements on the same patient scan, their geometric median was used to approximate the true corresponding landmark location.

Placements of landmarks on soft tissue for all patients deviated from their geometric median by a median 3D Euclidean distance of 3.0 mm (CT-CT), 5.6 mm (MRI-CT), and 2.6 mm (MRI-MRI). On bony anatomy, variability was significantly lower. Overall, variability was positively correlated with the deformation magnitude in the region.

There is large interobserver variability in anatomical landmark placements on pelvic CT and MRI scans. Variability frequently exceeds voxel size, challenging the AAPM guideline recommending landmark-based DIR quality assessment.