Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Data preparation and interannotator agreement: BioCreAtIvE task 1B.

Marc E Colosimo1, Alexander A Morgan, Alexander S Yeh

  • 1The MITRE Corporation, 202 Burlington Road, Bedford, MA 01730, USA. mcolosimo@mitre.org

BMC Bioinformatics
|June 18, 2005
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The Promise and Challenge of Large Language Models for Pharmacovigilance.

Drug safety·2025
Same author

Overview of the COVID-19 text mining tool interactive demonstration track in BioCreative VII.

Database : the journal of biological databases and curation·2022
Same author

Challenges and opportunities for mining adverse drug reactions: perspectives from pharma, regulatory agencies, healthcare providers and consumers.

Database : the journal of biological databases and curation·2022
Same author

Assessing Open-Ended Human-Computer Collaboration Systems: Applying a Hallmarks Approach.

Frontiers in artificial intelligence·2021
Same author

ADE Eval: An Evaluation of Text Processing Systems for Adverse Event Extraction from Drug Labels for Pharmacovigilance.

Drug safety·2020
Same author

Resilience of clinical text de-identified with "hiding in plain sight" to hostile reidentification attacks by human readers.

Journal of the American Medical Informatics Association : JAMIA·2020

Developing text mining tools for molecular biology requires high-quality gene lists. Our study highlights the importance of clear guidelines and answer pooling for accurate gene identification in model organisms like Fly, Mouse, and Yeast.

Area of Science:

  • Bioinformatics
  • Computational Biology
  • Molecular Biology

Background:

  • Text mining methods are crucial for extracting information from scientific literature.
  • Developing reliable training and test materials is essential for assessing these methods.
  • Automated gene identifier extraction from PubMed abstracts for model organisms (Fly, Mouse, Yeast) was the focus.

Purpose of the Study:

  • To prepare and evaluate training and test datasets for assessing text mining in molecular biology.
  • To assess the ability of automated systems to generate unique gene identifiers from PubMed abstracts.
  • To refine gene lists using manual annotation and answer pooling for improved accuracy.

Main Methods:

  • Generated normalized gene name lists from model organism databases.

Related Experiment Videos

  • Pruned gene lists for training data; refined with manual annotation for testing data.
  • Assessed data quality using interannotator agreement and answer pooling of participant results.

    • Main Results:

    • Interannotator agreement was high for Fly (87%) and Yeast (91%) but lower for Mouse (69%).
    • Answer pooling identified additional errors, particularly in Mouse, leading to significant updates in the Mouse gene list (8% change).
    • PubMed abstracts contain only a fraction of genes mentioned in full text (25% Fly, 36% Mouse).

    Conclusions:

    • Clear annotation guidelines and interannotator experiments are vital for validating gene lists.
    • Abstracts alone are insufficient for comprehensive gene identification.
    • Answer pooling is an efficient method for identifying conflicting genes and improving dataset quality.