Can Multi-Label Classifiers Help Identify Subjectivity? A Deep Learning Approach to Classifying Cognitive Presence in...

Area of Science:

• Educational Technology
• Artificial Intelligence in Education
• Natural Language Processing

Background:

• Manual categorization of cognitive presence in MOOC discussions faces challenges due to subjectivity.
• Existing single-label automatic classifiers may perpetuate this subjectivity.
• Understanding cognitive presence is crucial for enhancing online learning experiences.

Purpose of the Study:

• To investigate a multi-label deep learning approach for analyzing cognitive presence in MOOC discussions.
• To compare the performance of multi-label classifiers against traditional single-label methods.
• To explore the potential of multi-label classification in addressing subjectivity and multiplicity in cognitive presence analysis.

Main Methods:

• Developed and fine-tuned a BERT-based multi-label classifier.
• Trained classifiers on MOOC discussion messages categorized by expert coders.
• Employed a triangulation approach, comparing multi-label and single-label classifier performance.

Main Results:

• Multi-label classifiers slightly outperformed single-label classifiers.
• Multi-label classifiers predicted messages as one or two adjacent categories of cognitive presence, reducing errors compared to manual coding.
• Partial message categorization accuracy was achieved, indicating nuanced understanding.

Conclusions:

• Multi-label deep learning classifiers show potential for identifying research subjectivity in MOOC discussions.
• This approach can accommodate the multiplicity of cognitive presence categories within single messages.
• The findings contribute to a richer understanding of cognitive presence in online learning environments.