CogProg: Utilizing Large Language Models to Forecast In-the-moment Health Assessment

Forecasting future health status is beneficial for understanding health patterns and providing anticipatory support for cognitive and physical health difficulties. In recent years, generative large language models (LLMs) have shown promise as forecasters. Though not traditionally considered strong candidates for numeric tasks, LLMs demonstrate emerging abilities to address various forecasting problems. They also provide the ability to incorporate unstructured information and explain their reasoning process. In this paper, we explore whether LLMs can effectively forecast future self-reported health state. To do this, we utilized in-the-moment assessments of mental sharpness, fatigue, and stress from multiple studies, utilizing daily responses (N=106 participants) and responses that are accompanied by text descriptions of activities (N=32 participants). With these data, we constructed prompt/response pairs to predict a participant's next answer. We fine-tuned several LLMs and applied chain-of-thought prompting evaluating forecasting accuracy and prediction explainability. Notably, we found that LLMs achieved the lowest mean absolute error (MAE) overall (0.851), while gradient boosting achieved the lowest overall root mean squared error (RMSE) (1.356). When additional text context was provided, LLM forecasts achieved the lowest MAE for predicting mental sharpness (0.862), fatigue (1.000), and stress (0.414). These multimodal LLMs further outperformed the numeric baselines in terms of RMSE when predicting stress (0.947), although numeric algorithms achieved the best RMSE results for mental sharpness (1.246) and fatigue (1.587). This study offers valuable insights for future applications of LLMs in health-based forecasting. The findings suggest that LLMs, when supplemented with additional text information, can be effective tools for improving health forecasting accuracy.