An Interpretable Soft-Sensor Framework for Dissertation Peer Review Using BERT

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Educational Technology

Background:

• Graduate education faces challenges in quality monitoring due to complex, subjective dissertation evaluations.
• Existing automated analysis methods struggle with nuanced disciplinary criteria and lack interpretability for educators.

Purpose of the Study:

• To develop an interpretable soft-sensor framework for quantifying latent evaluation dimensions in dissertation reviews.
• To enhance automated analysis of peer-review texts using advanced natural language processing techniques.

Main Methods:

• Employed a BERT-based model with attention mechanisms for deep semantic modeling of dissertation reviews.
• Integrated Shapley Additive exPlanations (SHAP) to ensure model prediction interpretability and quantify characteristic importance.
• Developed an interpretable soft-sensor paradigm combining NLP with substantive review principles.

Main Results:

• The BERT-SHAP model significantly outperformed baseline methods in accuracy, precision, recall, and F1-score.
• The interpretability mechanism successfully identified key evaluation dimensions prioritized by experts.
• Demonstrated the model's effectiveness in providing actionable insights for dissertation improvement.

Conclusions:

• The developed framework offers a novel, interpretable approach to analyzing complex academic evaluations in the era of big data.
• This soft-sensor paradigm bridges NLP advancements with essential review principles, enhancing dissertation quality assurance.
• Provides a scalable and interpretable solution for educators and institutions to monitor and improve graduate education standards.