Settling the Score on Algorithmic Discrimination in Health Care

AbstractThe U.S. Office for Civil Rights recently issued a final rule under the Affordable Care Act that advances protections against algorithmic discrimination in health care by all clinical tools, not just AI models. The rule mandates that tools used in patient care must not discriminate based on race, color, national origin, sex, age, or disability. We believe that the tools covered by the rule should include existing clinical risk scores, calculators, and administrative applications, some of which have been developed for clinical settings without widespread evaluation. Often such tools perpetuate disparities and are adopted without thorough validation, posing risks to diverse populations. Currently, the U.S. Food and Drug Administration regulates clinical algorithms as "Software as a Medical Device" (SaMD), but it exempts SaMDs that are verifiable by clinicians, allowing some tools to bypass regulatory scrutiny. We argue that to ensure equity and safety in patient care, health care organizations must ensure that all tools, including historical risk scores, be comprehensively evaluated before they are used.