Monitoring of Artificial Intelligence in Hospitals

Area of Science:

• Healthcare technology
• Artificial intelligence in medicine
• Regulatory compliance

Background:

• Safe implementation of artificial intelligence (AI) algorithms in healthcare requires robust monitoring.
• Upcoming regulations mandate AI monitoring, necessitating an understanding of current capabilities and gaps.
• Hospitals face unique challenges, including technical debt, impacting the adoption of advanced monitoring solutions.

Purpose of the Study:

• To investigate the potential for monitoring AI systems within hospital environments.
• To review current regulatory requirements and the state-of-the-art in AI monitoring.
• To perform a gap analysis of ISO42001 against EU AI Act requirements for AI management systems.

Main Methods:

• Review of regulatory documents and existing AI monitoring technologies.
• Gap analysis of the ISO42001 standard, focusing on its alignment with the EU AI Act.
• Qualitative assessment of implementation challenges in hospital settings.

Main Results:

• Identified the necessity for comprehensive AI monitoring to detect input deviations, performance drifts, and unintended interactions.
• Highlighted significant gaps in current standards (ISO42001) when harmonized with the EU AI Act.
• Indicated practical implementation challenges related to data quality, infrastructure, and continuous improvement in hospitals.

Conclusions:

• Comprehensive AI monitoring is essential for safe and compliant AI deployment in hospitals.
• Existing standards require enhancement to fully address the complexities of AI in healthcare.
• Addressing technical debt and infrastructure limitations is critical for successful AI monitoring implementation in healthcare.