A testable framework linking diagnostic AI contribution to outcome measurement in clinical decision support

Area of Science:

• Health Services Research
• Artificial Intelligence in Healthcare
• Clinical Informatics

Background:

• Current evaluation of AI-CDSS focuses on model-centric metrics (e.g., sensitivity, specificity), which are insufficient for assessing real-world impact.
• Existing frameworks lack comprehensive measures for patient-relevant outcomes, workflow efficiency, economic performance, user experience, and equity.

Purpose of the Study:

• To propose a testable conceptual framework, the Diagnostic AI Contribution Score (DACS), linking AI-CDSS contribution to outcome prioritization and auditable measurement.
• To develop a methodological approach for selecting outcome domains, defining exposure-action-outcome linkages, and specifying necessary telemetry for AI-CDSS auditability.

Main Methods:

• A concept-driven structured narrative synthesis of literature across clinical AI evaluation, health services research, AI governance, and LLM deployment.
• Derivation of a DACS-informed framework for outcome domain selection, measurement design, and telemetry requirements.

Main Results:

• A six-domain taxonomy for AI-CDSS outcome metrics, distinguishing learning/governance from equity.
• A DACS-to-domain mapping heuristic with threshold-sensitivity logic.
• A four-step measurement framework with minimum telemetry and clinical implementation guardrails.
• LLM-specific extensions and clinical safety considerations (e.g., alert fatigue, automation bias).

Conclusions:

• The proposed DACS framework offers a structured, hypothesis-generating model for proportional outcome measurement and governance of AI-CDSS.
• Empirical validation of DACS scoring, domain mappings, and the impact of standardized telemetry is needed for improved AI-CDSS attribution, monitoring, and accountability.