Learning regular expressions for clinical text classification.

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Clinical Informatics

Background:

• Traditional NLP applications rely on manually created regular expressions.
• Manual creation is time-consuming and limits scalability in text classification.
• Automating regular expression generation is crucial for efficient NLP.

Purpose of the Study:

• To develop a novel algorithm for automated regular expression discovery (RED).
• To implement and evaluate RED-based text classifiers for clinical data.
• To compare the performance of RED classifiers against traditional methods like Support Vector Machine (SVM).

Main Methods:

• Designed a Regular Expression Discovery (RED) algorithm.
• Implemented two RED-based classifiers: RED+ALIGN and RED+SVM.
• Evaluated classifiers on clinical datasets (SMOKE, PAIN) using 10-fold cross-validation.

Main Results:

• RED classifiers achieved 80.9-83.0% accuracy, outperforming SVM by 1.3-3% (p<0.001).
• Consistent improvements in precision, recall, and F-measure were observed with RED classifiers.
• RED+ALIGN significantly corrected SVM misclassifications, improving accuracy on challenging instances.

Conclusions:

• Machine-generated regular expressions are effective for clinical text classification.
• RED-based classifiers offer a viable alternative to manual methods.
• Combining RED with other classifiers, such as SVM, enhances overall classification performance.