Human in the loop: Balancing artificial intelligence, clinical judgment, and legal responsibility

Area of Science:

• Psychiatry and Law
• Computational Social Science

Background:

• Artificial intelligence (AI) is increasingly used in mental health care, legal decision-making, and forensic practice.
• AI is argued to improve prediction, efficiency, and consistency in professional judgment.
• However, AI's predictive function differs fundamentally from human reasoning, understanding, and contextualization.

Purpose of the Study:

• To examine the tensions arising from AI integration in mental health and legal contexts.
• To highlight the limitations of purely algorithmic decision-making.
• To propose a framework for navigating AI integration while maintaining human oversight.

Main Methods:

• Analysis of four central tensions: epistemic, ethical, relational, and legal.
• Introduction of the HUMAN framework (Human judgment, Understand the model, Monitor performance, Accountability, Narrative).
• Emphasis on human reasoning and ethical responsibility in AI-supported practice.

Main Results:

• AI integration presents significant epistemic, ethical, relational, and legal challenges.
• Purely algorithmic decision-making is insufficient for clinical and legal contexts.
• The HUMAN framework offers a structured approach to manage AI integration.

Conclusions:

• Meaningful human oversight is critical for AI in mental health and law.
• Technological innovation should support, not undermine, professional foundations.
• Human professionals retain ultimate responsibility, emphasizing compassion, empathy, and moral reasoning.