Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Jul 3, 2026

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
07:50

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts

Published on: September 20, 2018

What Do Biomedical NER and Entity Linking Benchmarks Measure? A Corpus-Centric Diagnostic Framework.

Robert Leaman1, Rezarta Islamaj1, Zhiyong Lu1

  • 1National Library of Medicine, Bethesda, MD.

Arxiv
|July 2, 2026
PubMed
Summary

Analyzing biomedical corpora for named entity recognition (NER) and entity linking (EL) is crucial. Our framework reveals significant differences in corpus properties, impacting benchmark reliability and generalization.

Related Concept Videos

Methods of Documentation V: CBE01:23

Methods of Documentation V: CBE

Charting by Exception, or CBE, is a method of documentation used in healthcare, particularly in nursing, that focuses on documenting only significant or abnormal findings rather than recording every detail. This approach aims to streamline the documentation process, improve efficiency, and ensure that healthcare providers can quickly identify deviations from normalcy in patient assessments.
In CBE, healthcare professionals establish predefined standards of practice that define what constitutes...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

LMOD+: A Comprehensive Multimodal Dataset and Benchmark for Developing and Evaluating Multimodal Large Language Models in Ophthalmology.

ACM transactions on computing for healthcare·2026
Same author

Knowledge-guided contextual gene set analysis with large language models.

Bioinformatics (Oxford, England)·2026
Same author

Large Language Models Meet Biomedical Knowledge Graphs for Mechanistically Grounded Therapeutic Prioritization.

ArXiv·2026
Same author

DeepER-Med: Advancing Deep Evidence-Based Research in Medicine Through Agentic AI.

ArXiv·2026
Same author

MedHopQA: A Disease-Centered Multi-Hop Reasoning Benchmark and Evaluation Framework for LLM-Based Biomedical Question Answering.

ArXiv·2026
Same author

Toward Multimodal Conversational AI for Age-Related Macular Degeneration.

ArXiv·2026

Area of Science:

  • Computational linguistics
  • Bioinformatics
  • Natural Language Processing

Background:

  • Biomedical Named Entity Recognition (NER) and Entity Linking (EL) rely heavily on annotated corpora for benchmarking.
  • The actual utility and characteristics of these corpora for benchmarking are often assumed rather than rigorously evaluated.

Purpose of the Study:

  • To introduce a corpus-centric framework for diagnosing benchmark-relevant properties directly from corpus annotations and metadata.
  • To systematically analyze and compare properties of existing biomedical corpora used for NER and EL tasks.

Main Methods:

  • Developed a framework organizing standardized statistics into five families: scale/density/label distribution, lexical/conceptual structure, train-test overlap, metadata composition, and terminology coverage.
  • Applied the framework to nine diverse biomedical corpora (diseases, chemicals, cell types).

Related Experiment Videos

Last Updated: Jul 3, 2026

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
07:50

A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts

Published on: September 20, 2018

  • Utilized open-source code and an interactive dashboard for analysis and reproducibility.

    • Main Results:

    • Corpus properties vary substantially even for ostensibly similar tasks, affecting evaluation signals and generalization demands.
    • Significant differences were observed in train-test overlap, representation of biomedical literature, and concept space coverage.
    • Commonly reported corpus statistics (e.g., size, entity type) are insufficient for fully characterizing benchmark evaluations.

    Conclusions:

    • Corpus-centric diagnostics offer a practical approach to analyze corpora beyond surface descriptors.
    • The framework aids in identifying potential transfer risks and interpreting the scope of benchmarking conclusions.
    • Understanding corpus properties is essential for reliable biomedical NER and EL benchmarking.