Diagnostic Accuracy of a Mobile AI-Based Symptom Checker and a Web-Based Self-Referral Tool in Rheumatology:...

Area of Science:

• Rheumatology
• Medical Informatics
• Artificial Intelligence in Healthcare

Background:

• Diagnosis of inflammatory rheumatic diseases (IRDs) is often delayed due to non-specific symptoms and rheumatologist shortages.
• Digital diagnostic decision support systems (DDSSs) offer potential to expedite diagnosis and improve patient navigation.
• This study evaluates the diagnostic accuracy of AI-powered tools for IRDs.

Purpose of the Study:

• To assess the diagnostic accuracy of a mobile AI symptom checker (Ada) and a web-based tool (Rheport) for inflammatory rheumatic diseases (IRDs).

Main Methods:

• A prospective, multicenter, open-label, crossover randomized controlled trial was conducted with 600 new patients across 3 rheumatology centers.
• Participants used either Ada or Rheport for symptom assessment, with rheumatologist diagnosis as the gold standard.
• Primary outcome: correct identification of IRDs by DDSSs.

Main Results:

• 214 (35.7%) patients were diagnosed with an IRD; rheumatoid arthritis was most frequent (11.5%).
• Diagnostic accuracies for IRDs were 52% (Rheport), 63% (Ada D1), and 58% (Ada D5).
• Ada showed better accuracy for rheumatoid arthritis (D1: 42%, D5: 64%) but overall poor agreement between DDSSs.

Conclusions:

• The diagnostic accuracies of Ada and Rheport for IRDs were not promising in this patient population.
• DDSSs may lead to inefficient use of limited healthcare resources.
• Stringent regulation and significant improvements are necessary for DDSS safety and efficacy.