Contrastive learning-based video quality assessment-jointed video vision transformer for video recognition

Area of Science:

• Computer Vision
• Machine Learning
• Medical Informatics

Background:

• Video quality is crucial for accurate video classification.
• Blurred videos hinder the classification of conditions like Mild Cognitive Impairment.
• Video Quality Assessment (VQA) can potentially improve video classification.

Purpose of the Study:

• To propose a novel method, SSL-V3, combining Self-Supervised Learning-based Video Vision Transformer with No-reference VQA for enhanced video classification.
• To address the challenge of limited labeled data in VQA datasets.

Main Methods:

• Developed SSL-V3, integrating VQA into video classification using a Combined-SSL mechanism.
• Used video quality scores to directly tune feature maps for classification.
• Employed supervised classification tasks to optimize VQA parameters, linking both tasks.

Main Results:

• SSL-V3 demonstrated robust performance on two datasets.
• Achieved 94.87% accuracy in classifying interview videos within the I-CONECT healthcare dataset.
• Verified the effectiveness of the proposed method in improving video classification.

Conclusions:

• The proposed SSL-V3 method effectively enhances video classification by incorporating video quality assessment.
• The Combined-SSL mechanism successfully links VQA and classification, overcoming data limitations.
• SSL-V3 shows significant promise for applications in healthcare and beyond.