Why we need to maintain a critical view on big data and artificial intelligence predictions

Area of Science:

• Rheumatology
• Artificial Intelligence
• Medical Informatics

Background:

• Artificial intelligence (AI) and machine learning are promoted as transformative for medicine.
• Their practical impact in daily rheumatology is currently limited, despite high expectations.

Purpose of the Study:

• To examine the gap between AI expectations and reality in rheumatology.
• To identify methodological and conceptual limitations hindering AI adoption.

Main Methods:

• Review of historical parallels in medical research (e.g., vitamin D, microbiome).
• Analysis of meta-epidemiological work and conceptual limitations (reference class problem).
• Evaluation of methodological issues in AI models (small datasets, overfitting, validation).

Main Results:

• Early studies often show exaggerated effects that diminish with larger, higher-quality research.
• AI models in rheumatology frequently suffer from data limitations and poor validation.
• Real-world performance of AI tools can significantly differ from retrospective results.

Conclusions:

• AI can offer insights for specific tasks but cannot yet overcome noisy clinical data for individual prediction.
• Progress requires realistic expectations, large datasets, transparent methods, and rigorous validation.
• Focus should be on robust, interpretable tools for population and subgroup decision-making.