Tell me something interesting: Clinical utility of machine learning prediction models in the ICU

In recent years, extensive resources are dedicated to the development of machine learning (ML) based clinical prediction models for intensive care unit (ICU) patients. These models are transforming patient care into a collaborative human-AI task, yet prediction of patient-related events is mostly treated as a standalone goal, without considering clinicians' roles, tasks or workflow in depth. We conducted a mixed methods study aimed at understanding clinicians' needs and expectations from such systems, informing the design of machine learning based prediction models. Our findings identify several areas of focus where clinicians' needs deviate from current practice, including desired prediction targets, timescales stemming from actionability requirements, and concerns regarding the evaluation and trust in these algorithms. Based on our findings, we suggest several design implications for ML-based prediction tools in the ICU.