Cleft lip and palate-generative pre-trained transformer: A parameter-efficient, domain-specific language model for...

General-purpose large language models (LLMs) often lack specialized knowledge for cleft lip and palate (CLP), limiting clinical utility. Accessible, accurate, and affordable AI tools could potentially supplement existing educational resources for clinicians and patients.

To develop and evaluate CLP-generative pre-trained transformer (GPT), a parameter-efficient model fine-tuned for CLP, and to assess whether a smaller specialized model can achieve performance comparable to large generalist models for clinical deployment.

We built a CLP dataset of 7815 validated Q&A pairs from medical dialogs, literature, and guidelines. Using Qwen2-7B-Instruct as the base, we applied low-rank adaptation for domain adaptation. Performance was assessed via physician- and patient-oriented subjective evaluations and an objective test of 135 specialized multiple-choice questions.

CLP-GPT showed significant improvements over its baseline across all dimensions (p < 0.001). In physician evaluations, its accuracy surpassed Claude-3.5-Sonnet and Gemini-1.5-Pro (p < 0.05) and showed no significant differences from models like GPT-4o (p > 0.05). In patient evaluations, CLP-GPT performed comparably to leading large models across all dimensions (p > 0.05). On the objective test, it achieved the highest accuracy (71.85%), though differences among models were not statistically significant.

CLP-GPT, a parameter-efficient, domain-specific model, delivers performance comparable to other LLMs in CLP. This demonstrates that cost-effective, specialized models can achieve high performance without massive computational resources.