Regulating AI in Health Care: The Challenges of Informed User Engagement

The European Union's proposed Artificial Intelligence Act is a welcome, ambitious law on the regulation of AI systems. However, it underestimates the responsibilities placed on individual users to navigate the implementation of AI. Focusing on the health care sector, this policy piece examines challenges that the proposed law bypasses. First, effective human-AI collaboration in the diagnostic process hinges on the acknowledgment of AI's mediating role in this process, on forming a diagnostic dialogue between humans and AI. Second, with AI in this mediating role, the meaning of responsibility is changed to accommodate the broadened scope of clinician and patient duties, modified clinical workflows, and emergent medical norms. Finally, the challenge of media literacy concerns both the issues of access to knowledge and the ability to make informed choices regarding human-AI interaction. This policy piece suggests that embracing the complexity of the use practices is essential to achieving an effective human-AI partnership, in the medical sector and at large.