Related Experiment Video
Updated: Jul 30, 2025

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data
Published on: June 26, 2013
Impact of Professional Background on Inter-Annotator Variability and Accuracy During Annotation of Clinical Notes
Kristina Weishäupl1, Mario Schuster1, Christina Mayrl1
1University of Applied Sciences Wiener Neustadt, Wiener Neustadt, Austria.
Background:
The aging population's need for treatment of chronic diseases is exhibiting a marked increase in urgency, with heart failure being one of the most severe diseases in this regard. To improve outpatient care of these patients and reduce hospitalization rates, the telemedical disease management program HerzMobil was developed in the past.
Objective:
This work aims to analyze the inter-annotator variability among two professional groups (healthcare and engineering) involved in this program's annotation process of free-text clinical notes using categories.
Methods:
A dataset of 1,300 text snippets was annotated by 13 annotators with different backgrounds. Inter-annotator variability and accuracy were evaluated using the F1-score and analyzed for differences between categories, annotators, and their professional backgrounds.
Results:
The results show a significant difference between note categories concerning inter-annotator variability (p<0.0001) and accuracy (p<0.0001). However, there was no statistically significant difference between the two annotator groups, neither concerning inter-annotator variability (p=0.15) nor accuracy (p=0.84).
Conclusion:
Professional background had no significant impact on the annotation of free-text HerzMobil notes.
More Related Videos
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
06:48Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
Related Concept Videos
Methods of Documentation III: PIE
Naturalistic Observations
Errors occurring during blood pressure monitoring
Several factors...
The Professional Nurse
Communication skills: These are critical characteristics, especially speaking and listening.
Improving Translational Accuracy
Methods of Documentation VI: Case Management Model
For example, a patient with a chronic...