在临床实践中对机器学习模型的质量控制监测的考虑
Louis Faust1, Patrick Wilson1, Shusaku Asai1
1Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, United States.
JMIR medical informatics
|June 28, 2024
概括
在临床环境中维持机器学习 (ML) 模型的性能需要强有力的监测. 本研究提出了一个实用的平台和指导方针,用于将ML模型监测集成到医疗保健工作流程中,解决现实世界的实施挑战.
科学领域:
- 临床信息学是一种临床信息学.
- 机器学习在医疗保健中的应用
- 医疗AI的软件工程
背景情况:
- 临床实践中的机器学习 (ML) 模型需要持续的性能监测以确保有效性.
- 现有的文献往往侧重于性能下降的检测,不太强调更广泛的整合和维护挑战.
- 现实世界的部署需要实用解决方案来监测随时间推移的ML模型性能.
研究的目的:
- 详细说明在梅奥诊所开发和使用一个平台来监测生产级ML模型.
- 为将ML模型监控平台整合到技术基础设施和工作流程中提供考虑和指导方针.
- 记录与现实世界的实施和维护这些平台相关的经验,挑战和解决方案.
主要方法:
- 在六个月内开发了一个R Shiny应用程序作为监控平台.
- 记录了整合过程,重点关注可行性,设计,实施和政策考虑.
- 包括监控平台的源代码,以促进采用.
主要成果:
- 监测平台已经使用和维护了两年 (截至2023年7月).
- 确定了实施的四个关键支柱:可行性,设计,实施和政策.
- 突出了方法论性能变化检测之外的挑战,以成功实现现实世界的部署.
结论:
- 成功整合ML监控平台需要解决资源,设计,IT基础设施和政策等实际方面的问题.
- 开发的平台和记录的指导方针提供了一种实用的方法,以保持ML模型在临床环境中的有效性.
- 需要进一步的工作来解决ML监控解决方案的更广泛的实施和维护挑战.
相关概念视频
Data Validation
5.0K
Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...
Nursing assessment guides are generally based on holistic models rather than medical...
5.0K
Clinical Trials
6.7K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
6.7K
Errors occurring during blood pressure monitoring
687
Blood pressure monitoring is a crucial clinical procedure in diagnosing and managing various cardiovascular conditions. Despite its significance, the accuracy of blood pressure measurements can be compromised by multiple factors, potentially leading to either falsely high or low readings. These inaccuracies are critical as they can significantly impact patient care. So, it is vital to understand these challenges deeply and adopt strategic approaches to minimize errors.
Several factors...
Several factors...
687
Methods of Documentation VI: Case Management Model
565
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
565
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
125
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
125
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
48
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
48


