Federated TriNet-AQ: Explainable english proficiency classification in augmented and virtual reality learning

Area of Science:

• Immersive Learning Technologies
• Artificial Intelligence in Education
• Natural Language Processing

Background:

• Augmented Reality (AR) and Virtual Reality (VR) create dynamic language learning environments.
• Assessing language proficiency in these evolving settings presents significant challenges, including managing multimodal inputs and ensuring data privacy.
• Existing methods struggle with the complexity of learner interactions in immersive platforms.

Purpose of the Study:

• To propose TriNet-AQ, a federated, interpretable deep learning architecture for classifying English competency within AR/VR platforms.
• To address the challenges of multimodal learner input, model prediction interpretation, and user data protection in distributed systems.
• To enhance real-time, personalized language assessment in next-generation immersive learning environments.

Main Methods:

• Utilized Quantum Sinusoidal Encoding (QSE), Triaxial Attention Fusion (TAF), and Quantum Modulated Integration (QMI) for multimodal feature alignment and context-aware learning.
• Employed Hybrid Slime Gorilla Optimisation (HSGO) for accelerated convergence and improved performance.
• Implemented federated learning for decentralized training, enhancing user privacy and system flexibility.

Main Results:

• TriNet-AQ achieved 98.5% accuracy, 0.95 AUC, and 0.89 EPES on real-world AR/VR instructional datasets, outperforming baseline models.
• Demonstrated effective generalization with only a 3.5% accuracy drop on new data.
• SHAP-based interpretability confirmed obvious feature attributions and consistent user relevance, with statistical analysis (Cohen's d = 0.89, p < 0.001) confirming model significance.

Conclusions:

• TriNet-AQ offers a robust, interpretable, and private solution for real-time language evaluation in immersive learning.
• The model's advanced architecture and optimization techniques effectively handle the complexities of AR/VR language assessment.
• This approach paves the way for more effective and personalized language learning experiences in immersive technologies.