Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Domain-specific language models and lexicons for tagging.

Anni R Coden1, Serguei V Pakhomov, Rie K Ando

  • 1IBM, T.J. Watson Research Center, Hawthorne, NY 10532, USA. anni@us.ibm.com

Journal of Biomedical Informatics
|December 13, 2005
PubMed
Summary

Adding a small medical corpus to a general English one significantly improves part-of-speech tagging accuracy for medical texts. This approach enhances Natural Language Processing (NLP) model performance cost-effectively.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Multi-scale data improves performance of machine learning model for long COVID identification.

Communications medicine·2026
Same author

Governing real-world health data as a public utility.

Science (New York, N.Y.)·2026
Same author

LinkML: an open data modeling framework.

GigaScience·2025
Same author

Development of a robust corpus for automated evaluation of online health information in Chinese using the DISCERN scale.

Journal of the American Medical Informatics Association : JAMIA·2025
Same author

Advancing the science of genomic learning healthcare systems.

Learning health systems·2025
Same author

Mondo: integrating disease terminology across communities.

Genetics·2025

Area of Science:

  • Computational linguistics
  • Medical informatics

Background:

  • Part-of-speech (POS) tagging is crucial for Natural Language Processing (NLP) tasks like information retrieval and data mining.
  • High accuracy in POS tagging typically requires large annotated datasets.

Purpose of the Study:

  • To evaluate the effectiveness of domain-specific corpora in improving POS tagging accuracy for specialized domains.
  • To identify characteristics for quantifying corpus similarity to guide corpus creation for optimal POS tagger performance.

Main Methods:

  • Training POS tagger models using a large general-English corpus.
  • Augmenting the general corpus with a small, domain-specific (medical) corpus.
  • Evaluating tagger performance on medical domain test data.
  • Developing metrics to quantify corpus similarity.

Related Experiment Videos

Main Results:

  • A large general-English corpus alone yielded 87% accuracy for medical text POS tagging.
  • Augmenting with a small medical corpus boosted accuracy to over 92%.
  • Identified corpus characteristics that predict performance gains.

Conclusions:

  • Domain-specific data, even in small quantities, is essential for high-accuracy POS tagging in specialized fields.
  • The findings provide practical guidance for cost-effective corpus development for NLP applications in new domains.