Towards explainable language reasoning via multi-modal knowledge graphs

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Explainable AI

Background:

• Current large language models (LLMs) lack transparency in their reasoning processes.
• Opaque AI decision-making limits auditability, especially in critical applications.
• There is a need for interpretable AI systems that can explain their reasoning.

Purpose of the Study:

• To develop a framework for explainable language reasoning.
• To enhance the transparency and auditability of AI decision-making.
• To ground AI inference in explicit semantic relations using Multi-Modal Knowledge Graphs (MMKGs).

Main Methods:

• Proposed a framework for explainable language reasoning grounded in MMKGs.
• Unified textual, visual, and structural knowledge into a shared graph representation.
• Introduced a planner-executor design with an LLM generating symbolic plans and a graph engine executing them.

Main Results:

• The framework achieved competitive performance on six textual and multimodal benchmarks.
• Reached 79.8% Hits@1 on WebQSP and 49.3% accuracy on OK-VQA.
• Demonstrated an auditable explanation mechanism through replayable explanation subgraphs.

Conclusions:

• The MMKG-grounded reasoning architecture provides a transparent and auditable approach to AI decision-making.
• The framework clarifies theoretical claims regarding explanation faithfulness.
• This work advances the field of explainable AI by integrating multimodal knowledge for robust reasoning.