Using artificial intelligence for the development of a living evidence map: The pharmacopuncture example

Area of Science:

• Integrative and Complementary Medicine
• Health Informatics
• Artificial Intelligence in Research

Background:

• Evidence maps are crucial for visualizing research landscapes and identifying priorities, but traditional methods require extensive manual literature monitoring.
• Pharmacopuncture, a Korean medicine therapy, involves injecting medicinal extracts into acupoints, an area with a growing body of research.
• The need for efficient, up-to-date evidence synthesis in specialized fields like pharmacopuncture is significant.

Purpose of the Study:

• To develop and validate an artificial intelligence (AI)-based automated system for constructing and maintaining a living evidence map.
• To enhance the identification of research gaps and priorities in pharmacopuncture research through an automated system.
• To assess the performance and time efficiency of the AI system compared to manual methods.

Main Methods:

• A web-based system was developed, integrating PubMed API and Gemini AI for automated literature search, selection, data extraction, and classification.
• The system's accuracy was evaluated across nine distinct tasks, with expert manual review serving as the benchmark.
• An interactive bubble chart visualization system was implemented to facilitate intuitive identification of research gaps.

Main Results:

• The AI system demonstrated high overall accuracy (94.00%) in processing 202 articles, with specific tasks like sample size extraction achieving 0% error.
• Most tasks, including pharmacopuncture name extraction (22.22% error rate), exceeded 90% accuracy.
• A significant improvement in time efficiency was observed, with a 68.9% reduction in time required (59 minutes vs. 190 minutes), confirming the feasibility of daily updates.

Conclusions:

• The AI-driven living evidence map offers a substantial improvement over static methods for organizing research evidence.
• The system intuitively identifies research gaps, enabling more effective prioritization and direction of future pharmacopuncture research.
• The developed system provides accurate, continuously updated evidence monitoring for pharmacopuncture research, saving considerable time and resources.