Judgment aggregation, discursive dilemma and reflective equilibrium: Neural language models as self-improving...

Area of Science:

• Artificial Intelligence
• Computational Linguistics
• Epistemology

Background:

• Neural language models (NLMs) often exhibit output inconsistencies.
• Understanding the source of these inconsistencies is crucial for improving NLM reliability.

Purpose of the Study:

• To diagnose the causes of incoherence in neural language models.
• To propose and evaluate a novel remedy for NLM logical inconsistencies.

Main Methods:

• Training NLMs on synthetic text corpora generated by simulated artificial agents with individual belief systems.
• Utilizing social choice theory to model how NLM pre-training aggregates author judgments, leading to discursive dilemmas.
• Developing a self-training procedure inspired by reflective equilibrium to correct model inconsistencies.

Main Results:

• NLMs aggregate individual author judgments, leading to collective inconsistencies (discursive dilemmas) even with consistent individual beliefs.
• The proposed self-training procedure effectively reduces logical incoherence in NLM belief systems.
• The remedy corrects global mis-confidence and leads to epistemically superior belief states in NLMs.

Conclusions:

• Social choice theory provides insight into the origins of NLM inconsistencies.
• Epistemological principles offer a framework for resolving these inconsistencies, enhancing NLM reliability and logical coherence.