Relation extraction using large language models: a case study on acupuncture point locations.

Area of Science:

• Medical Informatics
• Computational Linguistics
• Traditional Chinese Medicine

Background:

• Accurate acupoint location is critical for acupuncture effectiveness.
• Large language models (LLMs) offer potential for extracting knowledge from medical texts.
• Existing methods for acupoint relation extraction may be limited.

Purpose of the Study:

• To evaluate LLMs for extracting acupoint location relations.
• To assess the impact of fine-tuning on GPT performance.
• To compare different LLMs including GPT-3.5, GPT-4, and Llama 3.

Main Methods:

• Utilized the WHO Standard Acupuncture Point Locations corpus (361 acupoints).
• Annotated five types of acupoint location relations (direction, distance, part of, near acupoint, located near).
• Compared pre-trained and fine-tuned GPT-3.5, pre-trained GPT-4, and pre-trained Llama 3.

Main Results:

• Fine-tuned GPT-3.5 achieved the highest micro-average F1 score of 0.92.
• Fine-tuned GPT-3.5 consistently outperformed other models across all relation types.
• LLMs demonstrated effectiveness in extracting complex spatial and relational information.

Conclusions:

• Domain-specific fine-tuning significantly enhances LLM performance for acupuncture relation extraction.
• LLMs can support clinical decision-making and educational tool development in acupuncture.
• Findings advance informatics applications in complementary medicine and NLP.