Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Steps in Outbreak Investigation01:18

Steps in Outbreak Investigation

644
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
644

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Humanoid robots in the operating room: a framework for staged integration of embodied AI in surgery.

NPJ digital medicine·2026
Same author

Initial specialist validation of clinical decision support recommendations from a machine learning-enabled digital cognitive assessment.

Frontiers in neurology·2026
Same author

Large reasoning models as thinking machines for medicine.

Nature biomedical engineering·2026
Same author

Pancreatic Cancer AI: The Need for Prospective Outcome Studies.

Radiology·2026
Same author

Multimodal foundation models exploit text to make medical image predictions.

Nature communications·2026
Same author

A universal foundation model for grounded biomedical image interpretation.

Nature communications·2026

Related Experiment Video

Updated: Feb 28, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K

Evaluation of Large Language Models as Emergency Department Revisit Predictors.

Emma Chen1, Luyang Luo2, Fatma Gunturkun3

  • 1Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA2Harvard John A. Paulson School Of Engineering And Applied Sciences, Cambridge, MA, USA, yingchen@g.harvard.edu.

Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing
|February 27, 2026
PubMed
Summary

Large Language Models (LLMs) show potential in clinical prediction but struggle with real-world emergency department (ED) revisit tasks. Current LLMs, even with advanced methods, do not outperform traditional models, highlighting limitations in clinical prediction.

More Related Videos

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
07:31

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Published on: May 15, 2020

8.2K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.7K

Related Experiment Videos

Last Updated: Feb 28, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
07:31

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Published on: May 15, 2020

8.2K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.7K

Area of Science:

  • Artificial Intelligence in Medicine
  • Clinical Informatics
  • Machine Learning for Healthcare

Background:

  • Large Language Models (LLMs) demonstrate capabilities in clinical reasoning and question answering.
  • The efficacy of LLMs for real-world clinical prediction tasks, such as forecasting patient outcomes, remains largely uninvestigated.
  • Predicting emergency department (ED) revisits is crucial for optimizing patient care and resource allocation.

Purpose of the Study:

  • To evaluate the performance of Large Language Models (LLMs) in predicting 30-day emergency department (ED) revisits.
  • To compare different LLM modeling paradigms, including direct prediction and embedding-based approaches.
  • To benchmark LLM performance against traditional machine learning models using comprehensive clinical data.

Main Methods:

  • A large-scale study involving 138,010 adult ED visits from Stanford.
  • Two LLM approaches were assessed: direct prediction with natural language output and embedding-based methods (LLM2Vec).
  • Retrieval augmentation was applied to direct prediction; LLM-derived embeddings were used for downstream modeling.

Main Results:

  • Embedding-based LLM approaches (LLM2Vec F1=0.4505) outperformed direct prediction methods (Claude 3.7 F1=0.4160 with retrieval augmentation).
  • Despite access to extensive data, all LLM approaches (F1=0.3022-0.4505) underperformed a traditional LightGBM model (F1=0.4614) using only structured data.
  • Analysis of 17,488 predictions suggested LLM reasoning failures include overweighting history, neglecting protective factors, and risk aversion.

Conclusions:

  • Current-generation LLMs exhibit fundamental limitations for clinical prediction tasks like ED revisit forecasting.
  • Embedding-based LLM methods show promise but do not yet surpass traditional models.
  • Further research is needed to address systematic reasoning failures in LLMs for clinical applications.