Data-hugging shields proprietary AI models from research that could disprove them.

Area of Science:

• Medical Artificial Intelligence (AI)
• Biomedical Signal Processing

Background:

• Commercial entities often withhold data, preventing independent verification of medical AI claims.
• Apple reported a mean absolute error of 2.9 years for age estimation using photoplethysmographic (PPG) signals.

Purpose of the Study:

• To independently assess the replicability of Apple's claimed age estimation accuracy using PPG signals.
• To investigate the reliability of medical AI claims in the absence of data transparency.

Main Methods:

• Utilized UK Biobank data for independent analysis.
• Evaluated age estimation models using PPG signals.
• Compared model performance against baseline predictions (e.g., predicting mean age).

Main Results:

• The claimed accuracy of 2.9 years mean absolute error for age estimation was found to be unreplicable.
• Independent analysis yielded results only marginally better than predicting the average age.
• The inherent noise in PPG signals challenges the feasibility of such high accuracy.

Conclusions:

• The opacity of proprietary datasets ('data hugging') obstructs scientific validation of medical AI.
• There is a critical need for curated, public benchmark datasets and standardized evaluation platforms.
• Protecting the public requires transparent and verifiable claims in the field of medical AI.