A Unified Understanding of Deep NLP Models for Text Classification

Area of Science:

• Natural Language Processing
• Machine Learning Interpretability
• Computational Linguistics

Background:

• Deep learning models for text classification are rapidly advancing.
• Existing methods lack a unified framework to explain diverse NLP models.
• A need exists for a measure explaining both low-level (word) and high-level (phrase) features.

Purpose of the Study:

• To develop a visual analysis tool, DeepNLPVis, for unified understanding of NLP text classification models.
• To provide quantitative explanations of information flow through model layers.
• To enable analysis from corpus to individual sample levels.

Main Methods:

• Developed DeepNLPVis, a visual analysis tool.
• Introduced a mutual information-based measure to quantify information maintenance at each model layer.
• Modeled intra- and inter-word information to assess word importance and relationships (e.g., phrase formation).
• Implemented multi-level visualizations: corpus-level, sample-level, and word-level.

Main Results:

• DeepNLPVis enables unified understanding of NLP models.
• The mutual information measure quantifies information retention across model layers.
• Multi-level visualizations facilitate analysis from dataset to individual instances.
• Case studies demonstrated effective identification of sample and architecture-related problems.

Conclusions:

• DeepNLPVis aids in understanding and improving NLP text classification models.
• The tool helps users identify and address issues in model architectures and training data.
• Provides a unified framework for analyzing and interpreting deep NLP models.