Convergence to Steady State in LLM-Generated Ontological Concepts

Area of Science:

• Computational Linguistics
• Bioinformatics
• Health Informatics

Background:

• Large Language Models (LLMs) are increasingly used for automated ontology expansion.
• LLM output quality is sensitive to sampling parameters, particularly 'temperature', which influences creativity versus factual accuracy.
• High temperatures can lead to 'hallucinations' or irrelevant concept generation.

Purpose of the Study:

• To investigate the relationship between LLM sampling temperature and the quality/consistency of generated ontology concepts.
• To test hypotheses regarding superset relationships and convergence speed at different temperatures.
• To analyze Environmental Determinants of Health (EnDOH) concepts using LLMs.

Main Methods:

• LLM concept generation using concept-structured prompting across temperatures from 0.1 to 1.0.
• Iterative execution of fixed prompts to assess convergence to a steady state (k-convergence).
• Analysis of generated concepts for superset properties and convergence rates.

Main Results:

• Contrary to the first hypothesis, higher temperatures did not consistently yield supersets of concepts generated at lower temperatures.
• The second hypothesis was supported: lower temperatures demonstrated faster convergence to a steady state.
• LLM sampling temperature significantly impacts concept generation stability and consistency.

Conclusions:

• The 'temperature' parameter in LLMs does not guarantee a simple superset relationship for ontology expansion.
• Lower sampling temperatures promote more stable and faster convergence in LLM-based concept generation.
• Careful selection of LLM parameters is crucial for reliable ontology development, especially in specialized domains like EnDOH.