ICMC: An Interpretable Cross-domain Multi-modal Classification model for grading teaching plan.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep neural networks (DNNs) excel at multi-modal classification but often lack interpretability, causing skepticism, especially in sensitive fields like education.
• This trust deficit hinders the adoption of DNNs in critical applications requiring transparent decision-making.

Purpose of the Study:

• To introduce an Interpretable Multi-modal Classification framework (ICMC) that boosts confidence and performance in DNNs for multi-modal tasks.
• To address the lack of interpretability in current DNNs, particularly for educational assessment.

Main Methods:

• ICMC employs a confidence-driven attention mechanism in intermediate layers to assess local and global information and detect anomalies.
• A confidence probability mechanism at the output layer uses both perspectives to enhance result certainty.
• New multi-modal datasets for automatic lesson plan scoring were created and released.

Main Results:

• ICMC achieved 2.5-6.0% higher accuracy and 3.1-7.2% greater F1-score than state-of-the-art models on educational and medical datasets.
• Reduced computational latency by 18% and demonstrated 15.7% superior cross-domain generalizability compared to transformer-based methods.
• Interpretability was confirmed via attention visualization and confidence scoring.

Conclusions:

• ICMC offers a robust solution for interpretable multi-modal classification, enhancing trust and performance in sensitive domains.
• The framework's generalizability and efficiency make it suitable for real-world applications beyond educational assessment.