ChartInsighter: An Approach for Mitigating Hallucination in Time-Series Chart Summary Generation With a Benchmark...

Area of Science:

• Data Visualization
• Artificial Intelligence
• Natural Language Processing

Background:

• Effective chart summarization is crucial for efficient data interpretation.
• Existing methods struggle with accuracy and semantic richness in time-series chart summaries.
• Hallucinations (inaccurate information) are a significant challenge in automated chart summarization.

Purpose of the Study:

• To introduce ChartInsighter, a novel system for automated time-series data chart summarization.
• To significantly reduce hallucinations in generated chart summaries.
• To establish guidelines for automatic generation by identifying key summary elements and hallucination types.

Main Methods:

• Utilizing a multi-agent approach for iterative summary generation and collaboration.
• Integrating external data analysis modules for enhanced insight extraction.
• Implementing a self-consistency testing method for validation and correction.
• Developing a high-quality benchmark dataset with annotated hallucination types.

Main Results:

• ChartInsighter demonstrates superior performance compared to state-of-the-art models.
• The system achieves the lowest summary hallucination rate.
• Evaluations confirm a significant reduction in various hallucination types.
• Overall summary quality is substantially improved.

Conclusions:

• ChartInsighter effectively addresses the challenge of hallucinations in time-series chart summarization.
• The multi-agent and self-consistency methods contribute to accurate and reliable summaries.
• The developed benchmark facilitates future research and evaluation in this domain.