Harnessing the Potential of LLMs in Hospital Management: Insights Into Medical Data Inquiry

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Database Management

Background:

• The increasing digitization of medical data necessitates advanced methods for analysis.
• Hospital operations expansion requires efficient data processing solutions.

Purpose of the Study:

• To evaluate the effectiveness of fine-tuned large language models (LLMs) for SQL query execution in a medical context.
• To compare the performance of open-source and proprietary LLMs for medical data analysis.

Main Methods:

• Integration of the QLoRA algorithm with LLMs (ChatGLM2-6b, Llama2-7b, Llama2-13b) for fine-tuning on a local SQL dataset.
• Application of Prompt-Engineering with ChatGPT for optimizing SQL query execution.
• Evaluation of zero-shot and few-shot performance of proprietary models (ChatGPT-3.5, GPT-4.1).

Main Results:

• Original open-source models exhibited minimal SQL generation capability.
• QLoRA fine-tuning substantially improved open-source model performance, with QLoRA-Llama2-13B achieving the highest score (EX 0.41).
• Few-shot GPT-4.1 demonstrated expert-level performance (EX 0.96), comparable to database engineers (EX 0.97).

Conclusions:

• Fine-tuned LLMs offer a robust framework for efficient medical data analysis.
• Few-shot GPT-4.1 significantly enhances SQL query execution for informed hospital decision-making.