How Trueness of Clinical Decision Support Systems Based on Machine Learning Is Assessed?

The application of machine learning algorithms in clinical decision support systems (CDSS) holds great promise for advancing patient care, yet practical implementation faces significant evaluation challenges. Through a scoping review, we investigate the common definitions of ground truth to collect clinically relevant reference values, as well as the typical metrics and combinations employed for assessing trueness. Our analysis reveals that ground truth definition is mostly not in accordance with the standard ISO expectation and that used combination of metrics does not usually cover all aspects of CDSS trueness, particularly neglecting the negative class perspective.