Evaluating Expert-Layperson Agreement in Identifying Jargon Terms in Electronic Health Record Notes: Observational...

Area of Science:

• Health Informatics
• Patient Communication
• Medical Terminology

Background:

• Patients struggle to understand medical jargon within electronic health record (EHR) notes, especially those with lower health literacy.
• Medical experts previously identified terms for a patient-facing medical jargon dictionary (NoteAid) based on their perception of patient need.

Purpose of the Study:

• To determine the level of agreement between medical experts and laypeople regarding what constitutes medical jargon in EHR notes.
• To assess the accuracy of medical experts in identifying jargon terms relevant to patients.

Main Methods:

• An observational study compared jargon identification in EHR notes between medical experts and laypeople recruited via Amazon Mechanical Turk.
• Participants reviewed 20 EHR sentences containing 325 expert-identified potential jargon terms.
• Data collected included layperson demographics, health literacy, and agreement metrics (Fleiss κ, Kendall rank correlation).

Main Results:

• Medical experts identified a significantly higher proportion of jargon (59%) compared to laypeople (25.6%).
• Good agreement was found among medical experts (κ=0.781), while fair agreement was observed among laypeople (κ=0.590).
• Medical experts demonstrated high sensitivity (91.7%) and specificity (88.2%) in identifying jargon when laypeople's ratings were the gold standard.

Conclusions:

• Discrepancies in jargon identification between experts and laypeople underscore the need for patient-centered definitions.
• Fair agreement among laypeople indicates diverse perspectives on what constitutes jargon, necessitating broad consideration.
• Medical experts can effectively identify jargon terms that are valuable for patient comprehension and annotation.