Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Types of Errors: Detection and Minimization01:12

Types of Errors: Detection and Minimization

12.1K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
12.1K
Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

7.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
7.4K
Nonsense-mediated mRNA Decay02:27

Nonsense-mediated mRNA Decay

12.2K
The Upf proteins that carry out nonsense-mediated decay (NMD) are found in all eukaryotic organisms, including humans. Each protein has an individual role, but they need to work in collaboration. Upf1 is an ATP-dependent RNA helicase that unwinds the RNA helix. Because Upf1 can unwind any RNA, Upf2 and Upf3 are required to help Upf1 discriminate between nonsense and normal mRNAs.
Usually, Upf3 binds to an Exon Junction Complex (EJC) at mRNA splice sites. If a ribosome fully translates the mRNA,...
12.2K
Non-equilibrium in the Cell01:16

Non-equilibrium in the Cell

5.7K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
5.7K
Automatic Processing and Automatic Social Behavior01:28

Automatic Processing and Automatic Social Behavior

357
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...
357

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Designing Emotion-Integrated Visualizations for Kidney Function Self-Management: User-Centered Design Study With Youth Transplant Recipients and Caregivers.

Journal of medical Internet research·2026
Same author

Speaker Role Identification in Clinical Conversations.

medRxiv : the preprint server for health sciences·2025
Same author

Amplifying Cultural Values with Collaborative Photo-Elicitation: Strengths-Focused Co-Design with Alaska Native People.

DIS. Designing Interactive Systems (Conference)·2025
Same author

Envisioning the future of primary care: intervention strategies to support patient-centered communication feedback technology.

Journal of the American Medical Informatics Association : JAMIA·2025
Same author

Patient and clinician acceptability of automated extraction of social drivers of health from clinical notes in primary care.

Journal of the American Medical Informatics Association : JAMIA·2025
Same author

Women's Educating and Coping Strategies for Cultivating Supportive Web-Based Spaces for Discussing Sexual and Reproductive Health: Co-Design Study.

Journal of medical Internet research·2025
Same journal

Evaluating the Evidence Base for New Mental Health Tech With APA Labs.

Journal of medical Internet research·2026
Same journal

Radiomics-Based AI for the Diagnosis and Prognosis of Vessels Encapsulating Tumor Clusters in Hepatocellular Carcinoma: Systematic Review and Meta-Analysis.

Journal of medical Internet research·2026
Same journal

Development and Validation of an Explainable Machine Learning Model to Assess the Prevalence Probability of Gastrointestinal Heat Retention Syndrome in Children: Cross-Sectional Study.

Journal of medical Internet research·2026
Same journal

Assessment of a Digital Health Platform Using Web Analytics and User Experience Measurements: Quantitative Study Based on RE-AIM.

Journal of medical Internet research·2026
Same journal

Sensor-Based Monitoring of Knee Osteoarthritis Symptoms in Free-Living Settings: Scoping Review.

Journal of medical Internet research·2026
Same journal

Effects of Immersive Virtual Reality Interventions on Symptom Management in Patients With Gastrointestinal Cancer: Systematic Review and Meta-Analysis of Randomized Controlled Trials.

Journal of medical Internet research·2026
See all related articles

Related Experiment Video

Updated: Apr 4, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K

Automatically Detecting Failures in Natural Language Processing Tools for Online Community Text.

Albert Park1, Andrea L Hartzler, Jina Huh

  • 1Department of Biomedical Informatics and Medical Education, School of Medicine, University of Washington, Seattle, WA, United States. alpark1216@gmail.com.

Journal of Medical Internet Research
|September 2, 2015
PubMed
Summary
This summary is machine-generated.

This study introduces a low-cost, automated method to identify errors in biomedical natural language processing (NLP) tools when analyzing patient-generated health text. The approach effectively detects common NLP failures, offering a scalable solution for assessing evolving tools.

Keywords:
UMLSautomatic data processinginformation extractionnatural language processingquantitative evaluation

Related Experiment Videos

Last Updated: Apr 4, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.3K

Area of Science:

  • Biomedical Informatics
  • Natural Language Processing
  • Health Informatics

Background:

  • Patient-generated health text is increasing but difficult to process with existing biomedical Natural Language Processing (NLP) tools.
  • Current NLP tools are often designed for clinical or researcher text, facing challenges with evolving technologies and vocabularies.
  • Manual annotation for NLP evaluation is time-consuming and resource-intensive, necessitating alternative assessment methods.

Purpose of the Study:

  • To explore a low-cost, automated approach for detecting failures in biomedical NLP tools processing patient-generated text.
  • To characterize common NLP failures in online health community text.
  • To demonstrate the feasibility of automated failure detection using MetaMap, a popular biomedical NLP tool.

Main Methods:

  • Manual review of 9657 online cancer community posts processed by MetaMap to characterize and categorize NLP failures.
  • Identification of 12 causes for inaccurate concept mappings across three failure types: boundary, missed term, and word ambiguity.
  • Development of automated methods combining NLP techniques and dictionary matching to detect identified failure types, followed by manual evaluation.

Main Results:

  • Characterized three primary failure types: boundary, missed term, and word ambiguity, with 12 underlying causes.
  • Automated methods detected nearly half of 383,572 MetaMap mappings as problematic.
  • Word sense ambiguity was the most frequent failure (82.22%), followed by boundary failures (15.90%) and missed term failures (1.88%).
  • Automated failure detection achieved high performance metrics: 83.00% precision, 92.57% recall, 88.17% accuracy, and 87.52% F1 score.

Conclusions:

  • Challenges in processing patient-generated health text with NLP tools were highlighted.
  • A feasible, low-cost automated approach for detecting NLP failures in patient-generated text was demonstrated.
  • The approach offers a scalable solution for continuously assessing and improving NLP tools and vocabularies for patient-generated health data.