使用统计过程控制进行分布外检测和放射性数据监测
Ghada Zamzmi1, Kesavan Venkatesh2,3, Brandon Nelson2
1Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, MD, 20993, USA. alzamzmigaa@fda.hhs.gov.
Journal of imaging informatics in medicine
|September 16, 2024
概括
本研究提出了一个新的框架,将机器学习和几何方法结合起来,用于检测OOD数据和监控ML模型中的数据漂移,以确保在现实应用中可靠的性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 医疗成像医学成像
背景情况:
- 机器学习 (ML) 模型在遇到其训练分布之外的数据时容易受到性能下降的影响.
- 数据漂移是支持机器学习的设备的关键问题,可能导致不可预测的结果.
- 有效的OOD检测和数据漂移监控对于强大的ML部署至关重要.
研究的目的:
- 引入一个新的OOD检测和数据漂移监控框架.
- 结合机器学习,几何方法和统计过程控制 (SPC).
- 评估框架在识别OOD图像和随着时间的推移监控数据流方面的有效性.
主要方法:
- 研究的特征提取和漂移量化用于OOD检测.
- 应用了一个整合ML,几何方法和SPC的框架.
- 评估了在区分各种医学成像模式和患者人口统计方面的表现.
主要成果:
- 获得了OOD图像检测的高灵敏度 (CT中的0.980,CXR中的0.984,儿科CXR中的0.854).
- 成功检测到数据流转移 (例如,CXR即时转换为非CXR,几天内CT转移) 的低虚假阳性率.
- 证明了框架的模式不可知性和模型独立性.
结论:
- 拟议的框架有效地检测OOD输入,并监测ML模型中的数据漂移.
- 它为各种成像模式和ML模型提供了可定制和广泛适用的解决方案.
- 这种方法在动态环境中提高了支持机器学习的系统的可靠性和安全性.
相关概念视频
01:15Introduction to Statistical Process Control
98
Statistical Process Control (SPC) is a method used to monitor and control quality within processes, particularly in manufacturing and service delivery, by employing statistical methods. SPC aims to distinguish between natural (common cause) variation and variation due to specific changes or events (special cause), allowing for timely improvements and sustained quality. The control chart, a pivotal tool in SPC, visually displays data over time alongside a central line of upper and lower control...
98
01:05Quality Control
153
Quality control is one of the three cyclical quality assurance activities that help keep a system under statistical control. Typical quality control activities include creating quality control charts, conducting proficiency testing, and documenting and archiving results.
Quality control helps track data, visualize trends, and identify variations, making it easier to detect deviations that may affect the accuracy of an analysis. One way to do this is by generating a quality control chart, which...
Quality control helps track data, visualize trends, and identify variations, making it easier to detect deviations that may affect the accuracy of an analysis. One way to do this is by generating a quality control chart, which...
153
01:02The R Chart
65
In statistical process control, control charts, particularly R charts, are instrumental in monitoring process variations and identifying non-random patterns that run charts might miss. R charts track the variability within process subgroups, which is crucial when standard deviation use is impractical or unknown process variations exist.
R charts are pivotal for pinpointing shifts in process variability. Stability is indicated when all data points remain within the defined upper and lower...
R charts are pivotal for pinpointing shifts in process variability. Stability is indicated when all data points remain within the defined upper and lower...
65
01:00Detection of Gross Error: The Q Test
5.7K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
5.7K
01:18Steps in Outbreak Investigation
114
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
114
01:15Data Validation
150
Method validation is a crucial process in analytical chemistry designed to confirm that a given method consistently produces reliable and high-quality results. This process is essential when a method is applied to different sample matrices or when procedural modifications are made, ensuring that the results meet acceptable standards across various applications.
Key parameters for method validation include:
Key parameters for method validation include:
150