Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Improving Translational Accuracy02:07

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Improving Translational Accuracy02:07

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Impression Management Techniques IV: Altercasting01:14

Impression Management Techniques IV: Altercasting

Altercasting is a strategic communication technique in which an individual imposes a specific identity or social role onto another person to influence their behavior and shape the interaction. By presuming a role—such as “responsible leader” or “patient person”—altercasting encourages the target to conform to that identity, often aligning their behavior with the expectations associated with the role. The power of this tactic lies in its subtlety; once a role is assigned, it becomes socially...
Pharmacodynamic Models: Overview01:27

Pharmacodynamic Models: Overview

Pharmacodynamic (PD) responses describe the interaction between a drug and its biological target, culminating in a physiological effect. These responses can be classified into different types: continuous variables, such as blood glucose levels; categorical outcomes, like survival rates; and time-to-event metrics, such as disease progression. Understanding and modeling PD responses are critical for optimizing drug efficacy and safety.PD models describe the relationship between drug concentration...
Statistical Software for Data Analysis and Clinical Trials01:12

Statistical Software for Data Analysis and Clinical Trials

Statistical software is pivotal in data analysis and clinical trials by providing tools to analyze data, draw conclusions, and make predictions. These software packages range from simple data management applications to complex analytical platforms, supporting various statistical tests, models, and simulation techniques. Their significance lies in their ability to handle vast amounts of data with precision and efficiency, enabling researchers to validate hypotheses, identify trends, and make...
Clearance Models: Physiological Models01:09

Clearance Models: Physiological Models

Drug clearance is a critical pharmacokinetic process involving the irreversible removal of drugs from the body through various organs over a specified time period. Physiological models are indispensable in determining organ-specific clearance, defined by the proportion of the drug eliminated per unit of time from the organ's blood volume.
The organ's clearance rate depends on the blood flow to the organ and the extraction ratio (E). The extraction ratio describes the organ's proficiency in drug...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

[Guidelines for the Transmission of Sensitive Medical Information via Secure Channels].

Revista medica de Chile·2025
Same author

Sex differences in work-related accidents extracted from free text in Spanish using natural language processing.

BMC public health·2025
Same author

NLP modeling recommendations for restricted data availability in clinical settings.

BMC medical informatics and decision making·2025
Same author

Developing and Validating an Automatic Support System for Tumor Coding in Pathology Reports in Spanish.

JCO clinical cancer informatics·2025
Same author

Response to Kempf et al on Methodological and Practical Aspects of a Distant Metastasis Detection Model.

JCO clinical cancer informatics·2024
Same author

A pseudonymized corpus of occupational health narratives for clinical entity recognition in Spanish.

BMC medical informatics and decision making·2024

Related Experiment Video

Updated: May 13, 2026

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
07:14

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models

Published on: December 23, 2025

Clinical-ShiftEval: a framework for simulating and evaluating model adaptation in dynamic clinical NLP tasks.

Fabián Villena1,2, Felipe Bravo-Marquez3, Jocelyn Dunstan4,5

  • 1School of Dentistry, Pontificia Universidad Católica de Chile, Santiago, Chile.

BMC Medical Informatics and Decision Making
|May 12, 2026
PubMed
Summary

Clinical natural language processing (NLP) models degrade with evolving data. A new framework, Clinical-ShiftEval, shows hybrid and in-context learning methods significantly improve model robustness against these real-world clinical shifts.

Keywords:
Artificial intelligenceConcept driftConcept evolutionNatural language processing

Related Experiment Videos

Last Updated: May 13, 2026

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
07:14

Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models

Published on: December 23, 2025

Area of Science:

  • Clinical Natural Language Processing (NLP)
  • Machine Learning in Healthcare
  • Electronic Health Records (EHR) Data Analysis

Background:

  • Clinical NLP models are crucial for extracting information from EHRs, but often fail in dynamic clinical environments.
  • Evolving data distributions (e.g., new guidelines, diseases) cause significant performance degradation in existing models.
  • Current evaluation methods assume static data, failing to capture real-world model adaptation challenges.

Purpose of the Study:

  • To introduce Clinical-ShiftEval, a framework for simulating and evaluating NLP model adaptation in dynamic clinical settings.
  • To assess model performance under label-set incompatibility (LSI) and task definition evolution (TDE) using real clinical text.
  • To compare the effectiveness of continual training, in-context learning, and hybrid approaches for model adaptation.

Main Methods:

  • Developed Clinical-ShiftEval to operationalize LSI and TDE through controlled data transitions from real datasets.
  • Applied the framework to the Chilean Waiting List Corpus, modeling LSI with referral specialty classification and TDE with disease prioritization.
  • Systematically compared continual training, in-context learning with large language models, and a hybrid adaptation strategy.

Main Results:

  • Clinical-ShiftEval reliably simulates realistic clinical changes and produces interpretable performance drops, validating its utility for benchmarking.
  • Conventional supervised models degraded significantly (up to 82% F1 drop for LSI, 43% for TDE).
  • In-context learning reduced performance drops (35% LSI, 10% TDE), while a hybrid method achieved ~8% drop; continual training required >30% new data to surpass these.

Conclusions:

  • Clinical-ShiftEval provides a robust method for benchmarking NLP model adaptation in dynamic EHR environments.
  • In-context learning and hybrid approaches offer substantial resilience to evolving clinical data without needing new labeled data.
  • Continual training with moderate new data is most effective for optimal accuracy recovery, eventually outperforming other methods.