Construction of performance score dynamic prediction system for clinical departments using explainable machine...

Accurate evaluation of clinical departmental performance is essential for public hospital management. However, existing approaches primarily rely on static, retrospective annual assessments and lack interpretability, limiting their ability to support early intervention and informed decision-making. To address these gaps, the study aimed to present a dynamic framework for predicting annual departmental performance score based on real-world hospital data using explainable machine learning.

Performance data between January 2023 to December 2024 was collected from the Hospital Information System (HIS). Six machine learning models, namely Linear Regression (LR), Decision Tree (DT), Random Forest (RF), Gradient Boosting, XGBoost, CatBoost, were trained to predict annual performance scores across three cumulative time windows (January-March, January-June, January-September) with traning set of 2023 and testing set of 2024. Model Evaluation was using metrics such as R², Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Mean Squared Error (MSE). To comprehensively enhance interpretability of dynamic model, SHapley Additive exPlanations (SHAP) analysis was applied to reveal key performance indicators across different time windows in predicting annual departmental performance. The web prediction System was developed using streamlit based on best-performing model.

A total of 648 records covering 24 months was obtained from 27 clinical departments. Compared with among models, LR model achieved the highest R² values (0.8964, 0.9314, 0.9640) and the lowest RMSE (0.0257, 0.0209, 0.0151), MAE (0.0190, 0.0162, 0.0116), and MSE (0.0007, 0.0004, 0.0002) across all stages. Through SHAP analysis, the top 5 contributing performance indicator were consistent across all stages, including proportion of medical service revenue in total medical revenue, proportion of outpatient revenue from medical insurance fund, proportion of consumables in total medical revenue, proportion of inpatient revenue from medical insurance fund, and average inpatient expense per admission. For clinical practice, the web tool, ClinDeptPredictor, was developed based on LR model, accesible at https://clindeptpredictor-tool.streamlit.app/.

The study presented a novel dynamic framework for clinical departmental performance prediction in tertiary public hospital using explainable machine learning. The framework supports progressive performance monitoring across multiple time windows and provides timely insights to inform managerial decisions. In addition, the development of the web tool helps facilitate practical application, offering methodological support and practical reference for performance management in public healthcare settings.