Predicting Outpatient Appointment Demand Using Machine Learning and Traditional Methods

Traditional methods have long been used for clinical demand forecasting. Machine learning methods represent the next evolution in forecasting, but model choice and optimization remain challenging for achieving optimal results. To determine the best method to predict demand for outpatient appointments comparing machine learning and traditional methods, this retrospective study analyzed "appointment requests" at a major outpatient department in a destination medical center. Two separate locations (A and B) were assessed with 20 traditional, hybrid (traditional + machine learning) and machine learning methods to determine the best forecasting outcome (lowest Forecast Standard Error, FSE). Data characteristics from both datasets were examined. 20 forecasting models were then assessed and compared for the best result. Location A's data displayed a cyclical and non-trending pattern while Location B's displayed a cyclical and trending pattern. Both Location A and B yielded the feature engineered XGBoost model (machine learning) with the lowest out-of-sample FSE. It is important to carefully analyze and understand the underlying data set pattern and then test a variety of traditional, machine learning, and hybrid prediction methods to achieve optimal predictive results. Additionally, the use of feature engineering or hybrid methods can augment the usefulness of machine learning methods.