Paired Validation of a Multimodal Vision-Transformer System for the Non-Invasive Diagnosis of Idiopathic Pulmonary...

Background/Objectives: Distinguishing idiopathic pulmonary fibrosis (IPF) from other interstitial lung diseases is prognostically and therapeutically important. We developed Fibresolve, a machine learning system that analyzes chest computed tomography (CT) to support IPF diagnosis. Fibresolve version 1 (v1) was validated in the PUFAIR study, met its co-primary endpoints, and received U.S. FDA authorization. Fibresolve version 2 (v2) was subsequently developed under the FDA's Predetermined Change Control Plan (PCCP) framework and, here, is evaluated against the authorized v1 model. Methods: v2 augments the original CT-only v1 classifier by incorporating age, sex, and forced vital capacity into an ensemble, while retaining the v1 CT-only model as a fallback when these variables are unavailable. Both models were evaluated in the same 300-patient PUFAIR cohort (83 IPF, 217 non-IPF) from two U.S. centers. Reference-standard diagnoses were established by institutional multidisciplinary discussion and supported by surgical pathology in 94.7% of cases. McNemar test, Cohen κ, and bootstrap noninferiority analyses were used for paired model comparisons. Results: In the key thin-slice diagnostic CT subgroup (N = 137), v2 achieved a sensitivity of 57.5% (95% CI, 42.2-71.5) and specificity of 84.5% (95% CI, 76.0-90.4), compared with 55.0% (95% CI, 39.8-69.3) and 82.5% (95% CI, 73.7-88.8) for v1. Performance improvements were directionally consistent across most subgroups, with no evidence of inferior performance. Conclusions: Fibresolve v2 preserved the performance of the FDA-authorized v1 model while providing modest, directionally favorable improvements. These findings support v2 as the primary model, with v1 retained as a robust fallback when clinical variables are unavailable.