Streamlining medical software development with CARE lifecycle and CARE agent: an AI-driven technology readiness level...

Area of Science:

• Medical Software Development
• Clinical Artificial Intelligence
• Machine Learning Systems

Background:

• Medical software development faces regulatory, ethical, and operational hurdles.
• Ensuring technical maturity and clinical safety is crucial for AI/ML deployment.
• Existing methods lack a comprehensive approach for clinical AI readiness.

Purpose of the Study:

• Introduce the Clinical Artificial Intelligence Readiness Evaluator (CARE) Lifecycle and Agent.
• Streamline technology readiness level assessments for medical AI software.
• Provide a framework for evaluating clinical AI/ML system maturity.

Main Methods:

• Iterative framework development with stakeholder analysis (clinical informatics, data engineers, ethicists, operational leaders).
• AI-driven tool using retrieval-augmented generation.
• Evaluation via a synthetic use case (Diabetes Outcome Predictor) with metrics on question coverage and review time.

Main Results:

• The CARE framework effectively addresses clinical AI development complexities.
• AI tool identified 32.8% unanswered assessment questions in a synthetic case.
• Human adjudication confirmed discrepancies in 19.4% of these instances, indicating potential for reduced manual review and accelerated timelines.

Conclusions:

• The CARE Lifecycle and Agent provide a robust method for evaluating medical AI maturity.
• Integration of stakeholder insights and AI assessment enhances security and effectiveness.
• Future work aims to optimize AI retrieval strategies and validate across diverse clinical applications.