Equiflow: An open-source software package for evaluating changes in cohort composition

Area of Science:

• Clinical Research Methodology
• Health Informatics
• Artificial Intelligence in Medicine

Background:

• Exclusion criteria and data preprocessing in clinical research can introduce hidden biases, affecting study validity and generalizability, especially in AI/ML.
• Traditional reporting often obscures how sample composition changes during participant selection.

Purpose of the Study:

• To introduce Equiflow, an open-source Python package for automated creation of enhanced participant flow diagrams.
• To quantify distributional shifts and visualize changes in key variables during participant selection.

Main Methods:

• Developed Equiflow, an open-source Python package.
• Automated the creation of participant flow diagrams tracking sample size and composition.
• Quantified distributional shifts and visualized variable evolution at each exclusion step.
• Applied Equiflow to a sepsis patient cohort from the eICU database.

Main Results:

• Sequential exclusions in a sepsis cohort reduced the sample size from 126,750 to 1,094 patients.
• Requiring non-missing troponin measurements caused significant, typically invisible, demographic shifts.
• Equiflow visualizations revealed substantial compositional biases introduced during cohort construction.

Conclusions:

• Equiflow enhances transparency in clinical research by making compositional biases visible before AI/ML modeling.
• Enables informed decisions on analyses and reporting of generalizability limitations.
• Supports the development of more equitable clinical AI systems in data-driven healthcare.