对人工智能模型的概括性进行决策区域分析:在交叉反应和人口转移的情况下估计模型的概括性
Alexis Burgon1, Berkman Sahiner1, Nicholas Petrick1
1U.S. Food and Drug Administration, Center for Devices and Radiological Health, Office of Science and Engineering Laboratories, Division of Imaging, Diagnostics, and Software Reliability, Silver Spring, Maryland, United States.
Journal of medical imaging (Bellingham, Wash.)
|January 29, 2024
概括
评估人工智能 (AI) 在医学成像中的通用性至关重要. 这项研究引入了一种分析AI决策空间的新方法,以改善概括性评估,特别是在有限的数据下.
科学领域:
- 医学成像人工智能 医学成像人工智能
- 机器学习的通用化能力
- 医疗保健中的人工智能
背景情况:
- 确保人工智能 (AI) 模型对目标人群进行概括对于安全的医疗器械部署至关重要.
- 传统的概括性评估需要广泛,多样化的数据集,通常无法在医学成像中使用.
- 现有的方法在有限的数据场景中扎,阻碍了可靠的AI评估.
研究的目的:
- 开发一种增强的方法来评估AI模型在医学成像中的通用性.
- 克服依赖于大,多样化的数据集的传统方法的局限性.
- 评估人工智能决策空间,而不仅仅是分布现有的测试数据.
主要方法:
- 通过测试图像三胞胎之间的插曲来生成虚拟样本的邻近分布.
- 杆测试设置特征以增加样本多样性,同时保持接近人工智能模型的数据组.
- 将方法应用于非临床的胸部X射线任务:患者的性别,种族,COVID状态和年龄组分类.
主要成果:
- 决策区域组成分析显示,在任务之间,单个"首选"类的决策空间不成比例.
- 尽管在评估数据集上表现相似,但人工智能模型显示偏差.
- 交叉反应性和人口转移分析表明,在代表不足的患者子组中,偏好类的预测过度.
结论:
- 分析人工智能模型的决策空间为其概括性提供了有价值的见解.
- 拟议的方法增强了概括性评估,特别是当测试数据有限时.
- 这种方法有助于识别潜在的偏见,并提高AI在医疗应用中的可靠性.
相关概念视频
Generalization, Discrimination, and Extinction
559
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
559
Intelligence
7.4K
The term "intelligence" is complex because it refers to both behavior and individuals, and its interpretation varies across cultures. European Americans tend to link intelligence with reasoning and cognitive skills, while in Kenya, it is tied to responsible participation in family and social life. In Uganda, intelligence is seen as the ability to know the right actions and carry them out effectively, while the Iatmul people of Papua New Guinea associate it with the capacity to remember...
7.4K
Case Studies
11.7K
There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it.
11.7K
Measures of Intelligence
7.2K
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
7.2K
Stereotype Content Model
14.7K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.7K
Decision Making: P-value Method
5.4K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
5.4K


