在本科医学教育中使用自然语言处理以通过叙事评估对学生进行分层的挑战
Laurah Turner1, Christine Yang Zhou2, Danielle Weber3
1Artificial Intelligence and Educational Informatics and Department of Medical Education, University of Cincinnati College of Medicine, Cincinnati, OH, United States.
概括
自然语言处理 (NLP) 模型在识别医学学生表现方面表现良好,但未能确定有风险的学生. 在NLP培训中包括"复制/粘贴"评论,提高了识别高表现和低表现的模型准确性.
科学领域:
- 医学教育 医学教育
- 自然语言处理自然语言处理.
- 学生的评估 学生的评估
背景情况:
- 医学教育工作者寻求有效的方法来解释叙事评估数据.
- 自然语言处理 (NLP) 提供了分析叙事数据以识别有风险的学生的潜力.
研究的目的:
- 根据叙事评估数据,评估NLP模型在识别有风险的医学学生方面的有效性.
- 调查"复制/粘贴"行为在叙事数据对NLP模型性能的影响.
主要方法:
- 利用了来自16个医学学生队伍 (2006-2022) 的叙事数据.
- 每个学生的T-score平均值 (TSA) 根据职工评估计算.
- 训练并测试了4个NLP模型,使用叙事数据和TSA来识别底部10%TSA的学生.
- 分析了"复制/粘贴"评论对模型准确性的影响.
主要成果:
- NLP模型的整体准确度为0.8,但无法可靠地识别TSA的底部10%的学生.
- 在叙事评价中发现了普遍的"复制/粘贴"行为.
- 将"复制/粘贴"评论纳入NLP模型提高了他们识别成绩最低和最高10%的学生的能力.
结论:
- 标准的NLP模型不足以准确识别有风险的医学学生.
- 叙事数据的歧视性实用性可能受到"复制/粘贴"行为的限制.
- 进一步的研究应该探索叙事数据中的词汇多样性和词量,并使用先进的大型语言模型进行分析.
相关概念视频
Case Studies
13.7K
There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it.
13.7K
Nursing Assessment
9.6K
The two sources for collecting information are primary and secondary. After gathering information, interpretation and validation help to complete the data. The purpose of assessment is to establish data with the initial information, to interpret data about the patient's perceived needs and health problems, and to respond to these problems identified.
The nurse collects all aspects of the patient's health in the initial assessment, establishing priorities for ongoing focused assessments...
The nurse collects all aspects of the patient's health in the initial assessment, establishing priorities for ongoing focused assessments...
9.6K
Classification of Illness
8.9K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
8.9K
Surveys
17.1K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
17.1K
Nursing Process for Patient and Caregiver Teaching I: Assessment and Diagnosis
2.5K
The nursing process provides a clinical decision-making framework for patients and families to establish and implement a personalized care plan. Since part of the nurse's duties is to teach patients, the steps of the nursing process are the most effective way to approach instruction. The nursing process and the teaching-learning process are inextricably linked.
It is critical to determine the patient's learning needs during the assessment. Determination of learning needs compounds data...
It is critical to determine the patient's learning needs during the assessment. Determination of learning needs compounds data...
2.5K
Stratified Sampling Method
15.6K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a stratified sample, divide the population into groups called strata and then take a...
To choose a stratified sample, divide the population into groups called strata and then take a...
15.6K

