Related Experiment Video
Updated: Jul 4, 2026
06:55
Inverse Probability of Treatment Weighting (Propensity Score) using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Assessing the Concordance Between LLM-Based ICD-10 Coding and a Drug Database for Antihypertensive Drug
Samya Adrouji1, Abdelmalek Mouazer2, Jean-Baptiste Lamy1
1Sorbonne Université, INSERM, Université Sorbonne Paris Nord, LIMICS, Paris France.
Studies in Health Technology and Informatics
|July 3, 2026
Summary
Large language models (LLMs) show substantial agreement with expert drug contraindication codes when they identify the same information. However, differences in data coverage limit overall entity-level concordance.
Area of Science:
- Pharmacology
- Medical Informatics
- Natural Language Processing
Background:
- Manual International Classification of Diseases, 10th Revision (ICD-10) coding for drug contraindications is labor-intensive and difficult to scale.
- Large language models (LLMs) show potential for clinical information extraction, but their reliability for structured pharmacological coding needs evaluation against expert databases.
Purpose of the Study:
- To evaluate the agreement between LLM-generated ICD-10 codes and the Thésorimed drug database for antihypertensive drug contraindications.
- To compare automated coding performance against a validated expert-curated resource.
Main Methods:
- Extracted contraindications from free-text Summary of Product Characteristics for 301 antihypertensive drugs using a retrieval-augmented generation pipeline.
- Aligned extracted contraindications with Thésorimed ICD-10 codes.
- Assessed agreement using binary kappa (κ1) for entity presence and hierarchical weighted kappa (κ2) for ICD-10 code concordance on 5,074 entities.
Main Results:
- Entity-level agreement was slight (κ1 = -0.30), indicating structural differences in source coverage.
- Code-level agreement for matched contraindications was substantial (κ2 = 0.70), with 76.9% identical ICD-10 codes.
Conclusions:
- LLM-based coding demonstrates substantial concordance with expert ICD-10 assignments when contraindications are identified by both sources.
- Significant differences in data coverage and representation between LLMs and expert databases were observed at the entity level.
Related Concept Videos
01:09Bioequivalence of Drugs: Drugs with Multiple Indications
The concept of therapeutic equivalence (TE) in drugs with multiple indications is complex. A generic drug may be therapeutically equivalent to a brand-name product for one specific indication, but this doesn't necessarily mean it's equivalent for all other indications. Evidence of TE in one patient group and bioequivalence shown in healthy volunteers can support—but not confirm—TE for other indications. However, definitive proof requires individual clinical studies for each indication due to...
01:20Drug Product Performance: In Vitro–In Vivo Correlation
In pharmaceutical development, it's crucial to establish a predictive in vitro–in vivo correlation (IVIVC) for two or more formulations to gain a comprehensive understanding of release properties. IVIVC reduces the need for costly in vivo studies and facilitates the establishment of meaningful dissolution specifications with significant cost savings and decreased regulatory burden. Furthermore, a meaningful IVIVC should predict Cmax and AUC within 20%, aligning with FDA guidance while adhering...
01:21Drug Dosing: Obese Patients
In the United States, obesity is a prominent concern. It is linked to heightened mortality rates due to increased occurrences of conditions such as hypertension, atherosclerosis, coronary artery disease, and diabetes compared to nonobese individuals. A patient is classified as obese if their actual body weight surpasses the ideal or desirable body weight by 20%, based on Metropolitan Life Insurance Company data. Ideal body weights consider average weights and heights for males and females...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:20Kendall's Coefficient of Concordance
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects or...
01:14Agonism and Antagonism: Quantification
When drugs are administered, they can elicit either an agonist or antagonist effect on the body. Agonism occurs when a drug activates a specific receptor, triggering a biological response. On the other hand, antagonism happens when a drug binds to the same receptors but blocks their activation, thereby preventing a biological response.
To quantify these effects, researchers use a dose-response curve, which provides valuable information about the potency and efficacy of a drug. Potency refers to...
To quantify these effects, researchers use a dose-response curve, which provides valuable information about the potency and efficacy of a drug. Potency refers to...