Participatory Design of AI-Based Clinical Decision Support Systems: Scoping Review

AI-based clinical decision support systems (CDSS) can improve diagnostics and treatment decisions, but they are rarely implemented in practice. Barriers include limited integration into clinical workflows, lack of transparency, and insufficient involvement of end users in system design. Participatory and user-centered approaches offer ways to address these challenges by aligning development processes with the needs and routines of clinical staff. However, systematic evidence on how such approaches are applied in the development of AI-based CDSS remains limited.

Our study examined how participatory approaches are used in the development, piloting, and implementation of AI-based CDSS. We analyzed which user perspectives were included, which participatory methods were applied, how they contributed to technical design, and which ethical, legal, and social implications were addressed.

This scoping review followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guideline, with a protocol published in advance. A systematic search was conducted in MEDLINE, ACM Digital Library, CINAHL, and PsycInfo for studies published from 2012 onward, complemented by snowballing and manual searches. Primary studies in English or German were included if they involved clinical staff in the development, piloting, implementation, or evaluation of AI-based CDSS. Moreover, 3 independent reviewers conducted screening and data extraction, resolving disagreements by consensus. Data analysis followed JBI methodology and focused on the scope of participation, theoretical and methodological foundations, and reported impacts of participatory approaches.

Of 4318 identified records, 37 met the inclusion criteria. The studies showed broad variation in terminology and methods, most often describing user-centered or iterative processes and less frequently co-design. Physicians were involved in nearly all studies, nurses frequently, and other professional groups only occasionally. Participation mainly supported requirements analysis, adaptation of models to clinical workflows, and the design of explainable interfaces. In several projects, it also influenced data selection, annotation, and visualization. Common barriers included time constraints, limited continuity of participation, and uncertainty toward AI. Ethical, legal, and social aspects were addressed implicitly through themes such as autonomy, responsibility, and traceability, while fairness and bias were rarely discussed.

Participatory processes in AI-based CDSS development should extend across all stages of system design and address not only usability but also data quality, bias, and broader ethical, legal, and social issues. Equal inclusion of nursing and therapeutic expertise is essential to reflect the diversity of clinical decision-making. Clear methodological standards are needed to ensure comparability and to strengthen participation as a genuine co-design process shaping data, models, and values in clinical AI.