探索使用ChatGPT来分析学生课程评价评论
Kathryn A Fuller1, Kathryn A Morbitzer1,2, Jacqueline M Zeeman1
1UNC Eshelman School of Pharmacy, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
BMC medical education
|April 19, 2024
概括
像ChatGPT这样的人工智能 (AI) 工具可以有效地分析学生的课程评估,显示与教师发现的高度一致,并显著减少分析时间. 虽然有益,但人工智能应该补充,而不是取代在教育评估中的人类判断.
科学领域:
- 教育技术的教育技术
- 教育中的人工智能
- 高等教育评估 高等教育评估
背景情况:
- 人工智能 (AI) 工具,特别是ChatGPT,在高等教育中提出了新兴的应用.
- 人工智能有可能减少教师的工作量,特别是在耗时的任务中,例如分析开放式课程评估评论.
- 目前用于从课程评估中的定性反中提取见解的方法通常是劳动密集型的.
研究的目的:
- 调查ChatGPT在分析开放式学生课程评估评论中的有效性.
- 为了比较人工智能驱动分析与传统教师分析的时间效率.
- 评估教练所确定的主题与ChatGPT产生的主题之间的一致程度.
主要方法:
- 教师独立地使用预定义的提示符对学生课程评估评论进行了主题分析.
- 两个独立的ChatGPT用户帐户使用类似的提示分析了相同的评论.
- 对于这两种方法,计算了包括百分比一致性和所需时间在内的定量指标.
主要成果:
- 在教师识别的主题和ChatGPT生成的主题之间观察到很高的一致性 (与课程相关的主题:0.71-0.82).
- 聊天GPT分析明显更快,平均为10.50-12.50分钟,而教练的时间为27.50分钟.
- 对于大多数提示,ChatGPT帐户显示了比与教师达成协议更高的互评级可靠性.
结论:
- 聊天GPT展示了作为一个辅助工具在健康专业教育课程评估中分析定性数据的显著潜力.
- 人工智能工具为创建主题提供了一个时间效率高的替代方案,与人类分析有很大一致性.
- 建议使用人工智能作为补充工具,将其输出与专家判断相结合,而不是完全依赖它.
相关概念视频
Comparing Experimental Results: Student's t-Test
1.6K
The t-test is a statistical method used to compare the sample mean with a population mean or compare two means from two data sets. The test statistic is calculated from the standard deviation, mean, and number of measurements in the data set at a selected confidence interval and then compared to a table of critical values at this confidence level. If the test statistic is smaller than the critical value, the null hypothesis is accepted. In this case, we state that the difference between the...
1.6K
Student t Distribution
6.0K
The population standard deviation is rarely known in many day-to-day examples of statistics. When the sample sizes are large, it is easy to estimate the population standard deviation using a confidence interval, which provides results close enough to the original value. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
The Student t distribution was developed by William S. Goset (1876–1937) of the...
The Student t distribution was developed by William S. Goset (1876–1937) of the...
6.0K
Statistical Analysis: Overview
6.6K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.6K
Study Design in Statistics
8.0K
A study design is a set of techniques that allow a researcher to collect and analyze data from different variables defined for a specific research problem. Statistics is commonly for effective study design and more robust experiments,
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
8.0K
Self-Evaluation: Self-Enhancement and Self-Verification
5.2K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.2K
Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test
1.6K
In parametric statistics, two fundamental tests stand out for their utility and wide application: the Student's t-test and goodness-of-fit tests. These tests provide researchers with a robust method for drawing insights from data, testing hypotheses, and making informed decisions based on their findings.
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
1.6K


