Latent Chain-of-Thought for Visual Reasoning

Area of Science:

• Artificial Intelligence
• Computer Vision
• Natural Language Processing

Background:

• Chain-of-thought (CoT) reasoning is crucial for Large Vision-Language Models (LVLMs) interpretability and reliability.
• Existing training methods (SFT, PPO, GRPO) exhibit limitations in generalizing to unseen tasks and are susceptible to biased reward models.

Purpose of the Study:

• To develop a scalable training algorithm for LVLMs that enhances CoT reasoning capabilities.
• To improve the generalization, effectiveness, and interpretability of LVLMs on complex reasoning tasks.

Main Methods:

• Reformulated LVLM reasoning as posterior inference, utilizing amortized variational inference for scalability.
• Introduced a novel sparse reward function with diversity-seeking reinforcement learning for token-level CoT generation.
• Implemented a Bayesian inference-scaling strategy using marginal likelihood to efficiently rank rationales and answers, replacing costly search methods.

Main Results:

• The proposed method significantly enhances state-of-the-art LVLMs.
• Demonstrated superior performance across seven reasoning benchmarks, improving effectiveness and generalization.
• Achieved enhanced interpretability in LVLM reasoning processes.

Conclusions:

• The novel training algorithm effectively addresses limitations in current LVLM reasoning training.
• The approach provides a scalable and robust method for improving CoT reasoning in LVLMs.
• The method offers a promising direction for developing more reliable and interpretable AI systems.