ChartQA-X: Generating Explanations for Visual Chart Reasoning

Area of Science:

• Artificial Intelligence
• Data Visualization
• Natural Language Processing

Background:

• Effective data-driven decision-making relies on understanding complex chart information.
• Generating accurate and coherent explanations for charts remains a significant challenge.

Purpose of the Study:

• To develop a comprehensive dataset, ChartQA-X, for training models to generate detailed explanations for chart images.
• To evaluate the quality of model-generated explanations against human-written ones.

Main Methods:

• Created ChartQA-X dataset with 30,799 chart samples, questions, answers, and explanations across four chart types.
• Generated and selected explanations based on faithfulness, informativeness, coherence, and perplexity metrics.
• Conducted human evaluations with 245 participants to assess explanation quality.

Main Results:

• Model-generated explanations in ChartQA-X outperformed human explanations in accuracy and logic.
• Fine-tuning models on ChartQA-X led to significant improvements: up to 24.57 points in explanation quality and 18.96 points in question-answering accuracy.
• Models demonstrated strong performance on unseen benchmarks, with a 14.75 percentage point gain.

Conclusions:

• ChartQA-X dataset facilitates the development of advanced AI agents capable of explaining complex visual data.
• The integrated approach of providing explanations alongside answers enhances comprehension and user trust in AI responses.