In-context adaptation of VLMs for few-shot cell detection in optical microscopy.

Area of Science:

• Biomedical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Foundation vision-language models (VLMs) excel with natural images but are underexplored in biomedical microscopy.
• Large annotated datasets are often unavailable for microscopic image analysis, limiting traditional deep learning approaches.

Purpose of the Study:

• Investigate the utility of in-context learning with state-of-the-art VLMs for few-shot object detection in microscopy.
• Introduce and utilize the Micro-OD benchmark for evaluating VLM performance on cell type detection.

Main Methods:

• Systematic evaluation of eight VLMs under few-shot conditions using the Micro-OD benchmark.
• Comparison of VLM variants with and without implicit test-time reasoning tokens.
• Implementation of a hybrid Few-Shot Object Detection (FSOD) pipeline combining a detection head with VLM-based classification.

Main Results:

• Zero-shot performance was weak due to domain gap, but few-shot support consistently improved detection accuracy.
• Marginal performance gains were observed after six shots.
• Some reasoning variants showed task-specific gains, varying across models and settings.

Conclusions:

• In-context adaptation is a promising direction for VLM development in microscopy.
• The Micro-OD benchmark provides a reproducible testbed for advancing open-vocabulary detection in biomedical imaging.
• Further research is needed to enhance VLM utility for microscopic image analysis.