Demographic reporting in biosignal datasets: a comprehensive analysis of the PhysioNet open access database

Area of Science:

• Biomedical Informatics
• Health Data Science
• Medical AI Research

Background:

• The PhysioNet database (PND) is a critical resource for biosignal data used in algorithm development.
• Understanding participant demographics is essential for contextualizing algorithm results and identifying potential biases.
• Incomplete demographic reporting in datasets can hinder the equitable development of AI and machine learning tools in medicine.

Purpose of the Study:

• To analyze the reporting patterns and characteristics of demographic data within the PhysioNet database.
• To assess the completeness of reporting for key variables: race, ethnicity, sex/gender, and age.
• To identify potential biases stemming from demographic imbalances in biosignal datasets.

Main Methods:

• Systematic review of 181 unique datasets in the PhysioNet database (as of July 6, 2023).
• Analysis of reporting completeness for four key demographic variables: race, ethnicity, sex/gender, and age.
• Evaluation of geographical distribution and sex/gender representation within reported studies.

Main Results:

• Only 175 of 181 datasets involved human participants.
• Less than 7% of studies reported on all four key demographic variables.
• Sex/gender and age were reported more frequently than race and ethnicity.
• Studies predominantly featured male participants and were concentrated in North America, particularly the USA.
• Significant underreporting and imbalances in demographic data were observed.

Conclusions:

• Incomplete and imbalanced demographic reporting in the PhysioNet database poses risks of embedded bias in AI/ML algorithms.
• The findings highlight a critical need for standardized, comprehensive demographic reporting practices.
• Ensuring equitable representation in biosignal data is crucial for the responsible development and deployment of medical AI.