医学LLM的新型评估框架:将模糊逻辑和MCDM结合起来,用于医学关系和临床概念提取
A H Alamoodi1,2,3, Omar Zughoul4, Dianese David5
1Institute of Informatics and Computing in Energy, Universiti Tenaga Nasional, Kajang, Malaysia. alamoodi.abdullah91@gmail.com.
Journal of medical systems
|August 30, 2024
概括
本研究引入了一种新的多标准决策 (MCDM) 方法来评估医学大语言模型 (LLM). 该框架优先考虑"医疗关系提取"并将GatorTron S 10B在医疗保健AI应用中排名最高.
科学领域:
- 医疗保健中的人工智能
- 医学自然语言处理 医学自然语言处理
- 决策支持系统是什么?
背景情况:
- 大型语言模型 (LLM) 越来越多地用于医疗保健,需要强大的评估框架.
- 目前的评估方法缺乏全面的方法来评估医学LLM的有效性,安全性和道德.
- 人工智能在医学中的整合需要方法来处理数据的不确定性和多方面的标准.
研究的目的:
- 开发一个全面的多标准决策 (MCDM) 方法来评估医学LLMs.
- 建立一个框架,解决医疗数据和决策的不确定性和复杂性.
- 在临床环境中指导选择和实施合适的LLM.
主要方法:
- 使用扩展的模糊加权零不一致性 (FWZIC) 方法与p,q-quasirung orthopair模糊集 (p,q-QROFS) 进行标准加权.
- 采用多属性理想-真实比较分析 (MAIRCA) 方法来评估医学LLM.
- 整合了模糊逻辑原则来管理不准确的医疗数据和标准.
主要成果:
- 发现"医疗关系提取"标准 (0.504) 比"临床概念提取" (0.495) 更重要.
- 在六个评估的LLM中,GatorTron S 10B获得了最高的排名.
- 在六个替代方案中,GatorTron 90B排名最后.
结论:
- 拟议的MCDM框架有效地评估医学LLM,考虑有效性,安全性和道德合规性.
- 该研究为医疗保健专业人员提供了一个实用的工具,帮助他们做出有关LLM采用的明智决策.
- 这项研究有助于将人工智能负责任地整合到医疗保健中,从而有可能改善患者的治疗结果.
更多相关视频
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
15.9K
09:20Cloud-Based Phrase Mining and Analysis of User-Defined Phrase-Category Association in Biomedical Publications
Published on: February 23, 2019
8.7K
相关概念视频
Methods of Documentation VI: Case Management Model
564
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
564
Data Validation
5.0K
Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...
Nursing assessment guides are generally based on holistic models rather than medical...
5.0K
Classification of Illness
7.4K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.4K
Concepts of Health and Illness
14.3K
Health is a condition of the body, mind, and spirit where an individual remains free from illness. Similarly, wellness is an active state, including living a lifestyle that promotes physical, mental, and emotional health. Physical health is critical for the overall well-being and can be affected by lifestyle, activity level, diet, and behavior. The highest attainable standard of health is a fundamental and universal human right. Consider Lisa, a fifteen-year-old born with congenital...
14.3K
