Check-worthy claim detection across topics for automated fact-checking

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Computational Linguistics

Background:

• Automated fact-checking systems rely on claim check-worthiness detection to prioritize sentences for verification.
• Existing research often overlooks the challenge of identifying check-worthy claims across diverse and unseen topics.
• Performance degradation is a significant issue when applying models to new subject areas.

Purpose of the Study:

• To assess and quantify the challenge of detecting check-worthy claims for novel, unseen topics.
• To propose and evaluate the AraCWA model for mitigating performance decline in cross-topic claim check-worthiness detection.
• To enhance the ability of fact-checking systems to generalize to new domains.

Main Methods:

• Developed the AraCWA model incorporating few-shot learning and data augmentation techniques.
• Utilized a publicly available dataset of Arabic tweets spanning 14 distinct topics.
• Quantified the challenge of cross-topic claim check-worthiness detection and analyzed topic similarities.

Main Results:

• The proposed data augmentation strategy within AraCWA significantly improved performance across topics.
• Performance gains varied across different topics, highlighting domain-specific challenges.
• Analysis of semantic topic similarities suggested a potential metric for predicting topic difficulty.

Conclusions:

• The AraCWA model effectively boosts performance for detecting check-worthy claims on new topics.
• Data augmentation is a crucial component for improving cross-topic generalization in claim check-worthiness detection.
• Semantic similarity analysis can serve as a useful proxy for estimating the difficulty of unseen topics for this task.