Explicit Image Caption Reasoning: Generating Accurate and Informative Captions for Complex Scenes with LMM

Area of Science:

• Computer Vision
• Artificial Intelligence
• Sensor Technology

Background:

• Traditional image captioning struggles with complex scenes.
• Advancements in sensor tech and deep learning offer new possibilities.

Purpose of the Study:

• Introduce Explicit Image Captioning Reasoning (ECR) for complex scenes.
• Improve caption accuracy and informativeness using sensor data.

Main Methods:

• Developed ECR with an enhanced inference chain for sensor images.
• Utilized the optimized ICICD dataset for training.
• Fine-tuned TinyLLaVA to create the ECRMM model.

Main Results:

• ECR effectively processes sensor data for deeper semantic understanding.
• The ECRMM model demonstrated superior performance.
• ECR outperformed traditional image captioning methods.

Conclusions:

• ECR offers a robust solution for complex scene image captioning.
• Leveraging sensor data and enhanced reasoning improves caption quality.
• This approach advances the field of AI-driven image analysis.