Using empirically constructed lexical resources for named entity recognition.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Bioinformatics

Background:

• Creating annotated corpora for named entity recognition (NER) is expensive and raises privacy concerns.
• Existing small annotated corpora may lack sufficient data for precise statistical NER.
• Manual lexicon creation is time-consuming and may not capture all relevant entities.

Purpose of the Study:

• To evaluate the effectiveness of automatically generated features based on distributional semantics for machine-learning NER.
• To determine if these features can overcome limitations of small annotated corpora.
• To compare the performance of automatically generated features against manually constructed lexicons.

Main Methods:

• Implemented machine learning models for named entity recognition (NER).
• Generated and experimented with distributional semantic features: n-nearest words, support vector machine (SVM)-regions, and term clustering.
• Evaluated feature performance by comparing F-scores against a baseline system and manually constructed lexicons.

Main Results:

• The addition of the n-nearest words feature significantly improved the F-score compared to the baseline system.
• Automatically generated lexicons derived from unannotated text demonstrated effectiveness.
• These features showed utility in extracting concepts from both biomedical literature and clinical notes.

Conclusions:

• Automatically generated distributional semantic features, particularly n-nearest words, enhance NER performance.
• Empirically derived lexicons from unannotated text can effectively supplement or replace manually created ones.
• This approach offers a viable solution for NER in domains with limited annotated data, such as biomedical and clinical text.