C4DF: confidence calibrated contextual control for diverse and faithful image captioning

Image captioning models generate coherent descriptions that are linguistically fluent. Because of similarity-based training objectives, generated captions are repetitive. In most techniques that aim to generate diverse, human-like captions, strong language priors lead to visually unsupported entities. This paper proposes Confidence-Calibrated Contextual Control for Diverse and Faithful Image Captions (C4DF), a decoding-time framework that reduces hallucinated objects while maintaining diversity and fluency. Image-specific pseudo-context adaptively suppresses tokens guided by confidence calibration. Prefix token conditioning controls diversity, while confidence-calibrated token suppression during decoding improves grounding. Experiments on the benchmark dataset demonstrate an improvement in CIDEr from 46.96 to 51.69 while reducing the noun non-context rate by 9.48%. Qualitative analysis confirms this balance between diversity and faithfulness without compromising on linguistic fluency.