学生をナラティブアセスメントで階層化するために自然言語処理を使用する際の課題 医学教育の大学院でのナラティブアセスメント
Laurah Turner1, Christine Yang Zhou2, Danielle Weber3
1Artificial Intelligence and Educational Informatics and Department of Medical Education, University of Cincinnati College of Medicine, Cincinnati, OH, United States.
まとめ
自然言語処理 (NLP) モデルは,医学生のパフォーマンスを特定するのに十分なパフォーマンスを示したが,リスクのある学生を特定することに失敗した. NLPトレーニングに"コピー/ペースト"のコメントを加えたことで,高性能と低性能の両方を識別するモデルの精度が向上しました.
科学分野:
- 医療教育 医療教育について
- 自然言語処理 (Natural Language Processing) とは,自然言語処理で処理される言語のことです.
- 学生の評価について
背景:
- 医療教育者は,ナラティブ評価データを解釈するための効率的な方法を模索しています.
- 自然言語処理 (NLP) は,ナラティブデータを分析して,リスクの高い生徒を特定する可能性を秘めています.
研究 の 目的:
- ネラティブ評価データに基づいて,リスクの高い医学生を特定するNLPモデルの有効性を評価する.
- ネラティブデータにおける"コピー/ペースト"行動がNLPモデルのパフォーマンスに与える影響を調査する.
主な方法:
- 医学生の16コホート (2006年-2022年) のナラティブデータを活用した.
- 計算されたT-スコア平均 (TSA) は,事務職の評価に基づいて,各学生が.
- ネラティブデータとTSAを使用した4つのNLPモデルをトレーニングし,テストし,TSAの最下位10%の学生を特定しました.
- モデルの正確性に対する"コピー/ペースト"コメントの効果を分析した.
主要な成果:
- NLPモデルは全体的に0.8の精度を達成しましたが,TSAの最下位10%の学生を確実に特定することができなかった.
- 物語の評価における一般的な"コピー/ペースト"行動を発見した.
- NLPモデルに"コピー/ペースト"のコメントを組み込むことで,パフォーマンスの最下位と最上位10%の生徒を特定する能力が向上しました.
結論:
- 標準的なNLPモデルは,リスクの高い医学生を正確に特定するのに不十分です.
- ネラティブデータの差別的有用性は",コピー/ペースト"行動によって潜在的に制限されています.
- 後の研究では,語彙の多様性や語数量に関する記述データを探究し,分析のために高度な大型言語モデルを採用すべきである.
関連する概念動画
Case Studies
13.7K
There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it.
13.7K
Nursing Assessment
9.6K
The two sources for collecting information are primary and secondary. After gathering information, interpretation and validation help to complete the data. The purpose of assessment is to establish data with the initial information, to interpret data about the patient's perceived needs and health problems, and to respond to these problems identified.
The nurse collects all aspects of the patient's health in the initial assessment, establishing priorities for ongoing focused assessments...
The nurse collects all aspects of the patient's health in the initial assessment, establishing priorities for ongoing focused assessments...
9.6K
Classification of Illness
8.9K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
8.9K
Surveys
17.1K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
17.1K
Nursing Process for Patient and Caregiver Teaching I: Assessment and Diagnosis
2.5K
The nursing process provides a clinical decision-making framework for patients and families to establish and implement a personalized care plan. Since part of the nurse's duties is to teach patients, the steps of the nursing process are the most effective way to approach instruction. The nursing process and the teaching-learning process are inextricably linked.
It is critical to determine the patient's learning needs during the assessment. Determination of learning needs compounds data...
It is critical to determine the patient's learning needs during the assessment. Determination of learning needs compounds data...
2.5K
Stratified Sampling Method
15.6K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a stratified sample, divide the population into groups called strata and then take a...
To choose a stratified sample, divide the population into groups called strata and then take a...
15.6K

