Hint recognition in Chinese and Russian diplomatic discourse using large language models

Area of Science:

• Computational Linguistics
• Pragmatics
• International Relations

Background:

• Diplomatic discourse analysis requires nuanced understanding of implicit communication.
• Existing Large Language Models (LLMs) struggle with high-context, multilingual implicit information recognition.

Purpose of the Study:

• To develop and evaluate an LLM system for hint recognition in Chinese and Russian diplomatic discourse.
• To integrate a semantic-cognitive-pragmatic framework with Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) reasoning for improved LLM performance.

Main Methods:

• Systematic annotation of press-conference transcripts from Chinese and Russian Ministries of Foreign Affairs.
• Construction of a vectorized external knowledge base for textual and logical hints.
• Integration of CoT reasoning and bilingual few-shot exemplars for LLM guidance.

Main Results:

• Stable overall performance with high recall across both Chinese and Russian datasets.
• Russian dataset showed higher precision and F1 scores compared to the Chinese dataset.
• Identified systematic biases including semantic over-interpretation and misclassification of hint types and literal meanings.

Conclusions:

• The study presents a feasible pathway for enhancing LLM accuracy in multilingual, high-context implicit information recognition.
• Proposed improvements include expanding negative samples, strengthening context anchoring, and implementing pre-filtering mechanisms.
• The findings offer practical guidance for improving LLM stability and accuracy in specialized discourse analysis.