Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Multiple Comparison Tests01:13

Multiple Comparison Tests

4.0K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
4.0K
Documentation in Long-Term and Home Healthcare Setting01:29

Documentation in Long-Term and Home Healthcare Setting

985
Documentation in long-term care facilities and home healthcare settings is crucial for ensuring continuous, coordinated, and comprehensive care for patients. Each setting has its specific documentation processes and tools:
Long-Term Care Facilities
985
Methods of Documentation VI: Case Management Model01:15

Methods of Documentation VI: Case Management Model

628
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
628
Health Information Technology and Healthcare Information System01:30

Health Information Technology and Healthcare Information System

917
Health Information Technology (HIT)
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
917
Nursing Clinical Information System01:27

Nursing Clinical Information System

865
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
865
Methods Of Healthcare Delivery System01:26

Methods Of Healthcare Delivery System

3.5K
At the different levels of the healthcare system, we see varying methods of healthcare used. These methods include managed care systems, case management, and primary healthcare.
Managed Care System:
The managed care system is designed to control the cost while maintaining the quality of care. The patient's care from admission to discharge is planned by the primary care provider or the case manager, also known as the gatekeeper. In a managed care system, the number of care providers is...
3.5K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Extracting Medical Information From Unstructured Clinical Text Using Large Language Models to Enhance Health Care Interoperability: Proof-of-Concept Study.

Journal of medical Internet research·2026
Same author

Recent Advances in Automated Mitosis Detection in Digital Pathology: A PRISMA-Guided Systematic Review with Evaluation-Regime Stratification (2018-2025).

Biomedicines·2026
Same author

Automated Pretreatment Thoracic CT-Based Body Composition Analysis Predicts Progression-Free Survival in Head and Neck Cancer.

Journal of clinical medicine·2026
Same author

Multimodal phenotypic classification of generalized anxiety and panic using structural MRI data and psychosocial factors: machine learning results from the German National Cohort (NAKO) study.

Translational psychiatry·2026
Same author

From Flow to Feature Using a Proof-of-Concept Spectral-Driven Machine Learning Approach Using Smart Urinary and Drainage Catheter Systems: Algorithm Development and Validation.

JMIR medical informatics·2026
Same author

Improving Retrieval Augmented Generation for Health Care by Fine-Tuning Clinical Embedding Models: Development and Evaluation Study.

Journal of medical Internet research·2026

Related Experiment Video

Updated: Sep 11, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

681

Using a Diverse Test Suite to Assess Large Language Models on Fast Health Care Interoperability Resources Knowledge:

Ahmad Idrissi-Yaghir1,2, Kamyar Arzideh2,3, Henning Schäfer2,4

  • 1Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Essen, Germany.

Journal of Medical Internet Research
|August 12, 2025
PubMed
Summary
This summary is machine-generated.

Large language models (LLMs) show strong performance on Fast Healthcare Interoperability Resources (FHIR) tasks, with commercial models like GPT-4o excelling. However, converting clinical notes to FHIR remains a challenge for all models.

Keywords:
DeepSeekGPT-4LLMshealth data interoperabilitylarge language modelsmedical informatics

More Related Videos

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

578
Constructing and Visualizing Models using Mime-based Machine-learning Framework
06:19

Constructing and Visualizing Models using Mime-based Machine-learning Framework

Published on: July 22, 2025

652

Related Experiment Videos

Last Updated: Sep 11, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

681
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

578
Constructing and Visualizing Models using Mime-based Machine-learning Framework
06:19

Constructing and Visualizing Models using Mime-based Machine-learning Framework

Published on: July 22, 2025

652

Area of Science:

  • Artificial Intelligence
  • Health Informatics
  • Natural Language Processing

Background:

  • Large language models (LLMs) demonstrate advanced capabilities in general knowledge but have limited evaluation within the specialized Fast Healthcare Interoperability Resources (FHIR) standard.
  • The complexity of FHIR poses challenges for LLMs trained on broad datasets, potentially limiting their understanding of domain-specific healthcare data.
  • Enhancing health data interoperability is crucial for leveraging clinical data and improving electronic health record interactions.

Purpose of the Study:

  • Introduce the FHIR Workbench, a novel dataset suite designed to rigorously assess LLM comprehension and application of the FHIR standard.
  • Evaluate the performance of both open-source and commercial LLMs on a variety of FHIR-related tasks.
  • Provide a benchmark for understanding LLM capabilities in healthcare data interoperability.

Main Methods:

  • Developed four distinct evaluation datasets to test FHIR knowledge, including multiple-choice questions on FHIR concepts and the FHIR Representational State Transfer (REST) API.
  • Assessed LLM performance in a zero-shot setting on tasks such as identifying FHIR resource types and generating FHIR resources from unstructured clinical notes.
  • Included human evaluations with six participants of varying FHIR expertise to contextualize LLM performance metrics.

Main Results:

  • Commercial models like GPT-4o achieved high F1-scores (e.g., 0.9990 on FHIR-ResourceID), while open-source models like DeepSeek-v3 also showed strong results (e.g., 0.9400 on FHIR-QA).
  • All evaluated LLMs struggled with the Note2FHIR task, indicating significant challenges in converting unstructured clinical text into FHIR-compliant resources, with scores ranging from 0.0382 to 0.3633.
  • Human participants achieved accuracy scores between 0.50 and 1.0 on the initial three FHIR tasks.

Conclusions:

  • Both open-source and commercial LLMs exhibit competitive performance on FHIR-related tasks, with commercial models currently leading in complex applications.
  • The FHIR Workbench serves as a critical tool for evaluating LLM capabilities and driving advancements in health data interoperability.
  • Continued development is needed to address the challenges LLMs face in accurately processing and converting unstructured clinical data into standardized FHIR resources.