Establishing best practices in large language model research: an application to repeat prompting

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Scientific Research Methodology

Background:

• Large language models (LLMs) are increasingly used in research.
• Ensuring the reliability and validity of LLM research is paramount.
• Repeat prompting is a common technique in LLM studies, but its statistical implications require careful consideration.

Purpose of the Study:

• To highlight the importance of establishing best practices in large language model research.
• To illustrate how ignoring correlations in repeated prompting data can affect study outcomes.
• To demonstrate the critical need for appropriate statistical methods in LLM analysis.

Main Methods:

• Utilized data from a previous study on potential model bias in medical abstract peer review.
• Compared statistical methods that ignore output correlation from repeated prompting with a random effects method that accounts for it.
• Analyzed intraclass correlation coefficient (ICC) to quantify within-group correlation.

Main Results:

• A high within-group correlation (ICC=0.69) was observed with repeated prompting.
• Ignoring this correlation led to over a 100-fold inflation of the effective sample size.
• Accounting for correlation reversed the study's findings from significant evidence of bias to no evidence of bias.

Conclusions:

• The establishment of best practices for large language model research is urgently needed.
• Accurate analysis of LLM research requires accounting for the correlation inherent in repeated prompting.
• Failure to implement best practices can critically impact the validity of research conclusions.