Related Experiment Video
Updated: May 15, 2025

A Knowledge Graph Approach to Elucidate the Role of Organellar Pathways in Disease via Biomedical Reports
Published on: October 13, 2023
Prompting large language models to extract chemical‒disease relation precisely and comprehensively at the document
Mei Chen1,2, Tingting Zhang1,2, Shibin Wang1,2
1Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE, Minzu University of China, Beijing 100081, China.
Large language models show promise for chemical-disease relation extraction, achieving high F1 scores. However, their effectiveness is limited by data scarcity and a tendency to misunderstand biomedical text nuances.
Area of Science:
- Biomedical Natural Language Processing
- Artificial Intelligence in Healthcare
- Pharmacovigilance
Background:
- Deep learning for chemical-disease relation extraction is hindered by limited annotated data.
- Existing methods struggle with precise relation typing and comprehensive extraction of related factors.
- Document-level extraction requires sophisticated understanding of complex biomedical texts.
Purpose of the Study:
- To evaluate the precise and comprehensive extraction capabilities of GPT3.5, GPT4.0, and Claude-opus for document-level chemical-disease relations.
- To analyze the impact of different prompting engineering workflows on extraction performance.
- To investigate content bias and error characteristics in large language model (LLM) extractions.
Main Methods:
- Development of six precise and five comprehensive extraction workflows using prompting engineering.
- Construction of a self-curated dataset for document-level chemical-disease relation extraction.
- Comparative analysis of LLM performance across different workflows and content types.
Main Results:
- LLMs achieved high F1 scores of 87% for precise extraction and 73% for comprehensive extraction.
- Prompting engineering strategies showed limited effectiveness, indicating LLM stubbornness in the extraction process.
- LLMs exhibited content bias, performing better on positive relations (e.g., induction, acceleration) and struggling with implicit meanings.
Conclusions:
- LLMs demonstrate significant potential for chemical-disease relation extraction, offering practical workflows.
- Optimizing training data is crucial for improving the accuracy and efficiency of future extraction methods.
- Addressing LLMs' interpretation of implicit biomedical text meanings is key to overcoming current limitations.
Related Concept Videos
Ligand Binding Sites
Protein-ligand interactions are quite specific; even though numerous potential ligands surround a cellular protein at any given time, only a particular ligand can bind to that protein. Moreover, a ligand binds only to a dedicated area on the surface of the protein, known as the...
Factors Influencing Drug Absorption: Disease States and Pharmacology
Substances such as alcohol and specific drugs, including antineoplastics, can also negatively impact drug absorption. For instance,...
Drug Discovery: Overview
Quantitative Aspects of Drug-Receptor Interaction
Factors Affecting Drug Response: Overview
Targets for Drug Action: Overview
Receptors are either membrane-spanning or intracellular proteins, which upon binding a ligand, get activated and transmit the signal downstream to elicit a response. Drugs bind receptors, either mimicking the action of endogenous ligands or blocking the receptor activity to bring about a modified response. Nearly 35% of approved drugs target the G...

