Conformity Between Values and EU Legal Requirements Regarding an AI Clinical Decision Support System (AI-CDSS) for...

The European Union Artificial Intelligence (AI) Act was adopted in 2024 to protect public interests from AI-related harms. We aimed to determine whether the AI Act captures the most relevant values of stakeholders (understood as any person potentially affected by the AI system, including, but not limited to, the direct users) related to a high-risk AI Clinical Decision Support System (CDSS) for cardiovascular disease. This system is intended to be implemented in primary care. A secondary analysis was carried out based on empirical data previously generated using ethics and design methodologies that had identified stakeholder values. These values were translated into the Trustworthy AI requirements of the High-Level Expert Group on AI and then contrasted with the AI Act. These requirements do not capture all identified stakeholder values. We describe three groups of untranslatable values and propose three additional complementary requirements. The AI Act mainly supports the requirements of human agency and oversight; transparency; and diversity, nondiscrimination, and fairness; but only partially, limited by its product safety nature. To address this gap, contractual strategies could provide a more legally robust ground for general practitioners to align the AI-CDSS with stakeholders' values, comply with their duty of care, and promote trustworthy AI-CDSSs.