COLA: Context-Aware Language-Driven Test-Time Adaptation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Test-time adaptation (TTA) addresses distribution shifts but often requires shared label spaces.
• Existing TTA methods are limited by the need for paired source and target domains with identical labels.
• Vision-language models (VLMs) show promise but struggle with capturing target domain specifics.

Purpose of the Study:

• To develop a general TTA method adaptable to multiple target domains without shared labels.
• To enhance VLM capabilities for effective domain-specific attribute recognition.
• To improve TTA performance and address class imbalance issues.

Main Methods:

• Proposed Context-aware Language-driven TTA (COLA) using a lightweight, context-aware module.
• Integrated task-aware, context-aware, and residual connection units with a frozen VLM.
• Introduced Class-Balanced Pseudo-labeling (CBPL) to handle class imbalance.

Main Results:

• COLA effectively adapts models to new domains without requiring shared labels.
• The method demonstrates strong performance in both TTA and class generalization tasks.
• The context-aware module offers parameter efficiency and minimal integration effort.

Conclusions:

• COLA provides a flexible and effective solution for TTA in more general scenarios.
• The approach enhances VLM utility by enabling domain-specific adaptation.
• COLA offers a promising direction for robust and adaptable machine learning models.