Training text chunkers on a silver standard corpus: can silver replace gold?

Area of Science:

• Biomedical Natural Language Processing (NLP)
• Computational Linguistics

Background:

• Training NLP chunkers for biomedical text requires annotated corpora, but creating gold standard corpora (GSCs) is costly and time-consuming.
• Existing GSCs are often small and domain-specific, limiting their utility.
• This study explores using automatically generated silver standard corpora (SSCs) as an alternative or supplement to GSCs.

Purpose of the Study:

• To evaluate the effectiveness of using SSCs for training chunkers in biomedical text processing.
• To compare two scenarios: training on an SSC in a new domain versus supplementing a small GSC with an SSC.

Main Methods:

• Utilized three chunking systems (Lingpipe, OpenNLP, Yamcha) and two corpora (GENIA, PennBioIE).
• Assessed performance in two scenarios: 1) training on an SSC in a new domain, 2) training on a GSC supplemented with an SSC.
• Evaluated noun-phrase and verb-phrase recognition performance using F-score metrics.

Main Results:

• Training on an SSC in one domain improved noun-phrase recognition performance compared to training on a GSC from a different domain.
• Supplementing small GSCs with SSCs significantly boosted chunker performance, achieving results comparable to much larger GSCs.
• Combined systems using SSCs showed limited improvement unless supplementing a GSC.

Conclusions:

• Silver standard corpora (SSCs) are a viable alternative or supplement to gold standard corpora (GSCs) for training biomedical NLP chunkers.
• Supplementing GSCs with SSCs is particularly beneficial for improving performance when GSCs are small.
• Further research is needed to determine the applicability of this approach to other NLP pipeline components.