在统计测试中评估大型语言模型ChatGPT,Claude,DeepSeek,Gemini,Grok和Le Chat的准确性和解释性质 选择假设测试决策的统计测试
Mukesh Shukla1, Deepshikha Pandey2, Samarjeet Kaur3
1Community Medicine, All India Institute of Medical Sciences, Raebareli, Raebareli, IND.
Cureus
|November 21, 2025
概括
大型语言模型 (LLM) 准确地选择用于假设测试的统计测试. 虽然所有模型都表现得完美,但它们的解释质量和统计推理能力存在差异.
科学领域:
- 生物统计学 生物统计学
- 人工智能的人工智能
- 科学研究方法学 科学研究方法学
背景情况:
- 大型语言模型 (LLM) 越来越多地用于研究,但它们选择用于假设测试的统计测试的能力尚未得到充分理解.
- 准确的统计测试选择对于科学有效性和避免错误结论至关重要.
研究的目的:
- 评估和比较六个著名的LLM在选择适当的统计测试中对假设测试场景的准确性.
- 评估这些LLM提供的解释和推理的质量.
主要方法:
- 六个LLM (ChatGPT,Claude,DeepSeek,Gemini,Grok,Le Chat) 的比较评估使用了20个标准化统计测试场景.
- 专家评估了模型响应在五个领域的准确性和质量:清晰度,假设识别,教学价值,问题解决和统计推理.
- 差异分析 (ANOVA) 用于进行统计比较.
主要成果:
- 所有六个LLM都在选择正确的统计测试时实现了100%的准确性.
- 在解释的质量上观察到显著的差异,克劳德在清晰度方面表现出色,而双子座在教学价值方面表现出色.
- 统计推理的质量各不相同,复杂回归分析的评分低于基本测试.
结论:
- 当前的LLM在假设测试中的统计测试选择中具有很高的准确性.
- 在解释质量和统计推理方面存在特定模型的优缺点.
- 研究人员可以依靠LLM进行统计咨询,但应该批判性地评估他们的输出,以获得详细的解释.
关键词:
聊天 聊天 聊天克劳德: 克劳德是我们的朋友.在深度搜索中,深度搜索.双子座 (Gemini) 是一个双子座.格洛克格洛克是什么意思一个假设就是假设.大型语言模型.李聊天聊天 李聊天聊天这是一个统计测试测试.更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
07:31Defining the Role Of Language in Infants' Object Categorization with Eye-tracking Paradigms
Published on: February 8, 2019
7.2K
相关概念视频
Accuracy and Errors in Hypothesis Testing
552
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
552
Statistical Hypothesis Testing
6.1K
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
6.1K
Testing a Claim about Standard Deviation
2.9K
A complete procedure to test a claim about population standard deviation or population variance is explained here.
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
2.9K
Errors In Hypothesis Tests
5.8K
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
5.8K
Decision Making: Traditional Method
5.0K
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
5.0K
Types of Hypothesis Testing
27.8K
There are three types of hypothesis tests: right-tailed, left-tailed, and two-tailed.
When the null and alternative hypotheses are stated, it is observed that the null hypothesis is a neutral statement against which the alternative hypothesis is tested. The alternative hypothesis is a claim that instead has a certain direction. If the null hypothesis claims that p = 0.5, the alternative hypothesis would be an opposing statement to this and can be put either p > 0.5, p < 0.5, or p...
When the null and alternative hypotheses are stated, it is observed that the null hypothesis is a neutral statement against which the alternative hypothesis is tested. The alternative hypothesis is a claim that instead has a certain direction. If the null hypothesis claims that p = 0.5, the alternative hypothesis would be an opposing statement to this and can be put either p > 0.5, p < 0.5, or p...
27.8K
