Benchmarking AI: comparing a fully automated spinal motion solution to a validated analyst-driven technology

Manual line drawings in spinal assessments are associated with poor accuracy and reproducibility, while many AI models lack validation. A semi-automated image analysis technology, Quantitative Motion Analysis (QMA), has been used to produce radiographic measures in over 300 clinical trials of spinal treatment and shown to have sub-degree and sub-millimeter accuracy (0.47° and 0.54 mm on average). Recently, a fully automated analysis pipeline of AI models has been developed to provide equivalent accuracy and reliability of QMA in clinical workflow. The objective of this study is to present a direct comparison between motion measurements obtained by a fully automated image analysis tool, SpineCAMP, to those obtained by experienced operators using a legacy semi-automated method, QMA.

About 120 lateral radiographs, evenly distributed across the cervical and lumbar spine regions with 30 flexion-extension exams per anatomical region, were collected from 49 different clinical sites and include preoperative and postoperative imaging. After excluding images with discrepant labeling (4/120, 3.3%), 66 cervical and 59 lumbar levels were analyzed. Each exam had segmental rotation and translation measured by 5 trained QMA analysts and SpineCAMP's fully automated AI pipeline.

SpineCAMP had consistent anatomy and image view with QMA in 96.7% of images. QMA and SpineCAMP demonstrated excellent agreement with vertebral landmark placement, (r² = 0.9999, 0.9999 for cervical landmarks X and Y positions; r² = 0.9996, 0.9997 for lumbar landmarks X and Y positions). Mean absolute errors between QMA and SpineCAMP for segmental rotation and translation were 0.16° and 0.09 mm in the cervical spine and 0.18° and 0.14 mm in the lumbar spine.

Spinal motion measures are commonly utilized to support clinical decisions. A fully automated, AI-driven technology can deliver strong agreement to validated analyst-driven technology, thereby offering clinicians a convenient solution for measuring spinal motion.