对生成人工智能的评估应包括重复来评估稳定性
Lingxuan Zhu1, Weiming Mou2, Chenglin Hong1
1Department of Oncology, Zhujiang Hospital, Southern Medical University, Guangzhou, China.
JMIR mHealth and uHealth
|April 30, 2024
概括
在医疗保健中评估像ChatGPT这样的生成人工智能 (AI) 需要重复,因为响应的可变性. 整合重复可以确保AI在医学中的可靠和可重复的研究成果.
科学领域:
- 人工智能在医学中的应用
- 医疗保健技术评估 医疗保健技术评估
- 医学研究方法学 医学研究方法学
背景情况:
- 像ChatGPT这样的生成人工智能模型对医疗保健应用非常有前途.
- 由于响应的可变性和随机性,评估这些模型存在挑战.
- 确保人工智能性能评估的可靠性至关重要.
研究的目的:
- 强调在评估生成性AI模型 (如ChatGPT) 时需要纳入重复的必要性.
- 突出响应变异性对研究结论的影响.
- 倡导在医疗保健中使用可靠和可重复的AI评估方法.
主要方法:
- 评论讨论了生成AI响应固有的随机性所带来的挑战.
- 它与生物实验进行并行,这些实验需要多次重复才能有效.
- 通过重复试验来评估AI稳定性的重要性得到了强调.
主要成果:
- 如果不考虑重复,可能会导致偏见的结论和研究可信度下降.
- 需要一致的评估方法来确保AI性能数据的可靠性.
- 该研究强调需要对评估方法进行透明报告.
结论:
- 重复对于可靠评估医疗保健中生成性AI模型至关重要.
- 研究人员必须从一开始就将重复纳入研究设计中.
- 对方法的透明报告提高了人工智能研究结果的稳定性和可重复性.
相关概念视频
Non-equilibrium in the Cell
4.4K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.4K
Generalization, Discrimination, and Extinction
540
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
540
Distribution Reliability and Automation
107
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
107
Random and Systematic Errors
10.9K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
10.9K
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K


