LLM-based prior elicitation for Bayesian graphical modeling.

Area of Science:

• Psychology
• Computational Statistics
• Network Science

Background:

• Bayesian graphical models use priors to represent assumptions about network structure.
• Specifying accurate hyperparameters for these priors is challenging due to limited theoretical guidance and simplistic default choices.
• Existing priors like Bernoulli, Beta-Bernoulli, and Stochastic Block have limitations in practical application.

Purpose of the Study:

• To introduce a novel framework for eliciting priors in Bayesian graphical models using large language models (LLMs).
• To convert LLM-generated inclusion judgments into specific prior probabilities and hyperparameters for network analysis.
• To provide an accessible R package (bgmElicit) with a Shiny app for implementing the LLM-based prior elicitation.

Main Methods:

• Developed an LLM-based prior elicitation framework.
• Integrated LLM inclusion judgments into Bernoulli, Beta-Bernoulli, and Stochastic Block priors.
• Created the `bgmElicit` R package with a Shiny application for user-friendly implementation.
• Validated the framework using a PTSD network meta-analysis subset and an empirical analysis of PTSD symptoms.

Main Results:

• LLM-based elicited priors can modestly strengthen evidence for edge presence and absence in psychological networks.
• OpenAI GPT models were compared across conditions on a PTSD network subset.
• The framework provides a complementary approach to expert judgment and prior sensitivity analyses.

Conclusions:

• LLM-based prior elicitation offers a practical solution to challenges in specifying hyperparameters for Bayesian graphical models.
• The `bgmElicit` package makes this advanced methodology accessible to researchers.
• This proof-of-concept work enhances the analysis of psychological network structures.