Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Types of Errors: Detection and Minimization01:12

Types of Errors: Detection and Minimization

11.6K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
11.6K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Self-organizing three-dimensional dermal papilla cell spheroids yield therapeutic extracellular vesicles that target hypertrophic scar regression via the miR-26a-5p/CCNE2 axis.

Burns & trauma·2026
Same author

Machine learning-guided predictive modeling and optimization of polyamide-based thin-film nanocomposite reverse osmosis membranes.

Environmental research·2026
Same author

A systematic evaluation of double-expressor lymphoma: prognostic impact, determinants of outcome, and comparative efficacy and safety of novel therapies.

Frontiers in oncology·2026
Same author

Impact of altitude and temperature variation on microbial communities and fermentation quality of natural oat and maize silage on the Qinghai-Tibet Plateau.

AMB Express·2026
Same author

From Concept to Clinic: Innovative Applications and Design Evolution of Microneedles in Wound Healing.

Advanced healthcare materials·2026
Same author

Retraction Note: Adipose tissue-derived stem cells suppress hypertrophic scar fibrosis via the p38/MAPK signaling pathway.

Stem cell research & therapy·2026

Related Experiment Video

Updated: May 5, 2026

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns
13:44

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns

Published on: August 30, 2013

42.6K

A Clinically-Informed Framework for Evaluating Vision-Language Models in Radiology Report Generation: Taxonomy of

Hao Guan1,2, Peter C Hou1,2, Pengyu Hong3

  • 1Brigham and Women's Hospital, Boston, MA.

AMIA ... Annual Symposium Proceedings. AMIA Symposium
|February 23, 2026
PubMed
Summary
This summary is machine-generated.

This study introduces a new framework to evaluate AI-generated radiology reports, focusing on clinical risks. It reveals common errors in vision-language models (VLMs), improving AI safety in medical imaging.

More Related Videos

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.3K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K

Related Experiment Videos

Last Updated: May 5, 2026

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns
13:44

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns

Published on: August 30, 2013

42.6K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.3K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K

Area of Science:

  • Medical Imaging AI
  • Natural Language Processing
  • Clinical Informatics

Background:

  • Vision-language models (VLMs) show promise for automatic radiology report generation.
  • Current evaluation metrics for VLMs in radiology are insufficient, lacking clinical specificity.
  • Existing methods rely on general NLP metrics or broad clinical scores, missing critical safety nuances.

Purpose of the Study:

  • To develop a clinically informed evaluation framework for VLM-generated radiology reports.
  • To establish a taxonomy of radiology-specific errors with associated clinical risk levels.
  • To introduce a novel risk-aware metric for quantifying the safety impact of AI-generated reports.

Main Methods:

  • Defined a 12-type radiology error taxonomy, annotated with physician-assigned clinical risk levels (low, medium, high).
  • Conducted error analysis on three VLMs (DeepSeek VL2, CXR-LLaVA, CheXagent) using 685 MIMIC-CXR cases.
  • Introduced the Clinical Risk-weighted Error Score for Text-generation (CREST) metric.

Main Results:

  • Identified critical vulnerabilities and common error patterns across evaluated VLMs.
  • Revealed condition-specific risk profiles associated with different types of AI-generated report errors.
  • Demonstrated significant differences in performance and safety impact among the tested models.

Conclusions:

  • The proposed framework provides a safety-centric foundation for evaluating medical report generation models.
  • Findings offer actionable insights for developing more reliable and clinically safe AI tools in radiology.
  • The CREST metric enables quantitative assessment of safety risks in VLM-generated reports.