Class-Sensitive TPB-Guided Memory Refinement for Online Zero-Shot Anomaly Detection

Zero-shot anomaly detection is attractive for industrial inspection, where target-domain training data are often unavailable for newly introduced products. Recent CLIP-based methods have demonstrated promising generalization, and online memory mechanisms can further improve adaptability by incorporating incoming test samples. However, unreliable or ambiguous evidence may be incorporated during online memory updates, which can degrade subsequent predictions, especially for weak or visually unstable categories. In this work, we propose TSMR, a lightweight extension of RareCLIP for online zero-shot anomaly detection. Rather than modifying the backbone or redesigning the anomaly scoring pipeline, TSMR improves the reliability of test-time memory evolution through a class-sensitive selective update strategy. Specifically, it combines a confidence quantile gate, a text-prior-based reliability check, and weak-class selective activation to derive a frame-level memory-update decision during online inference. Experiments on VisA and MVTec AD show that TSMR achieves clear improvements on VisA while maintaining competitive performance on MVTec AD. Under the online protocol, TSMR improves the reproduced RareCLIP baseline on VisA from 94.4% to 95.1% in image-level AUROC, from 98.8% to 98.9% in pixel-level AUROC, and from 93.5% to 94.0% in PRO. On MVTec AD, TSMR achieves 98.0% image-level AUROC, 97.6% pixel-level AUROC, and 93.6% PRO, remaining competitive with the strong RareCLIP baseline. Object-wise and seed-wise analyses further indicate that selective memory refinement is particularly beneficial for selected weak categories and remains stable across different online evaluation orders. These results suggest that reliable online memory evolution is an effective direction for CLIP-based zero-shot anomaly detection.