An interpretable method for automated classification of spoken transcripts and written text.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Machine Learning

Background:

• Distinguishing between spoken and written language is crucial for text classification.
• Deep neural networks (DNNs) like DistilBERT are common but often lack interpretability.
• Classical machine learning methods may offer more transparency.

Purpose of the Study:

• To compare text classification performance on spoken versus written language.
• To introduce and evaluate a novel, interpretable linear classifier.
• To assess the performance gap between classical and DNN-based methods.

Main Methods:

• Utilized radio show transcripts (spoken) and Wikipedia articles (written) for a new dataset.
• Developed a linear classifier with extensive n-gram features.
• Compared the novel classifier's accuracy and confidence measure against DistilBERT.
• Evaluated DistilBERT on fill-in-the-blank tasks for spoken and written text.

Main Results:

• The interpretable linear classifier achieved accuracy within 0.02 of DistilBERT.
• The proposed classifier includes an integrated confidence measure for reliability assessment.
• DistilBERT demonstrated similar performance on fill-in-the-blank tasks for both language types.
• The performance gap between classical and DNN methods can be significantly narrowed.

Conclusions:

• Classical text classification methods, with enhancements, can rival DNN performance.
• Interpretability is a critical factor in choosing classification methods for high-stakes decisions.
• The choice between methods depends on the necessity for transparency and explainability.