Harnessing the Potential of LLMs in Hospital Management: Insights Into Medical Data Inquiry

Considering the rapid expansion of hospital operations and the increasing digitisation of medical data, there is a pressing need for efficient and intelligent methods to process and analyse large-scale medical data.

We integrated the QLoRA algorithm with ChatGLM2-6b, Llama2-7b, and Llama2-13b models, fine-tuning them on a local SQL dataset to optimise query performance. Prompt-Engineering with ChatGPT was further applied for effective SQL execution.

Original open-source models showed almost no SQL generation capability (overall EX ≈ 0 for ChatGLM2-6B and Llama2-7B; 0.04 for Llama2-13B). QLoRA fine-tuning substantially improved performance, with QLoRA-Llama2-13B achieving the best results among open-source models (overall EX 0.41 ± 0.030). Proprietary models demonstrated significantly stronger performance. Zero-shot ChatGPT-3.5 achieved moderate accuracy (EX 0.44 ± 0.007), which improved to 0.94 ± 0.017 with few-shot prompting. GPT-4.1 further improved performance, reaching 0.78 ± 0.038 in the zero-shot setting and 0.96 ± 0.011 with few-shot prompting. The few-shot GPT-4.1 results were comparable to those of database engineers (EX 0.97 ± 0.017), with no significant difference (p = 0.42).

Fine-tuned LLMs and few-shot GPT-4.1 demonstrate substantial improvements in SQL query execution, providing a robust framework for efficient medical data analysis and informed hospital decision-making.