Adaptive cost-sensitive stance classification model for rumor detection in social networks

Area of Science:

• Computational Social Science
• Natural Language Processing
• Machine Learning

Background:

• The rapid growth of online social networks necessitates early detection of misinformation (rumors).
• Stance detection, analyzing user attitudes towards a rumor, is crucial for rumor detection.
• Stance detection is inherently imbalanced, with 'query' and 'deny' stances being rare, hindering classifier performance.

Purpose of the Study:

• To address the performance degradation in stance detection caused by imbalanced data.
• To propose a novel adaptive cost-sensitive loss function for deep neural networks to improve rare class detection.
• To enhance the accuracy of rumor detection systems by improving stance classification.

Main Methods:

• Developed a novel adaptive cost-sensitive loss function based on cross-entropy.
• The cost matrix within the loss function is adaptively tuned during training, not manually set.
• Applied the proposed method to stance classification on real-world Twitter and Reddit datasets (RumorEval 2017 & 2019).

Main Results:

• The proposed adaptive cost-sensitive loss function significantly improves the performance of stance classifiers in detecting minority classes.
• Demonstrated capability in detecting rare classes ('query', 'deny') on Twitter and Reddit data.
• Achieved substantial improvements in the mean F-score for rare classes: ~13% on RumorEval 2017 and ~20% on RumorEval 2019.

Conclusions:

• The novel adaptive cost-sensitive loss function effectively handles imbalanced stance detection data.
• The method enhances overall stance classification performance, particularly for underrepresented user stances.
• This approach offers a promising solution for improving the reliability of automated rumor detection systems.