Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpus

Area of Science:

• Medical Informatics
• Natural Language Processing
• Health Data Privacy

Background:

• De-identification of medical records is crucial for privacy and research.
• Existing datasets for de-identification research are limited.
• Longitudinal medical records present unique de-identification challenges.

Purpose of the Study:

• To create a de-identified corpus of longitudinal medical records for research.
• To establish a gold standard for medical record de-identification.
• To support the 2014 i2b2/UTHealth natural language processing shared task.

Main Methods:

• De-identification of 1304 longitudinal medical records from 296 patients.
• Application of a broad interpretation of Health Insurance Portability and Accountability Act (HIPAA) guidelines.
• Utilized double-annotation, arbitration, sanity checking, and manual proofreading.
• Automated replacement of private health information with realistic surrogates.

Main Results:

• Achieved an average token-based F1 score of 0.927 for annotator agreement with the gold standard.
• Developed the first corpus of its kind for de-identification research.
• Systems in the 2014 i2b2/UTHealth shared task achieved a mean F-measure of 0.872 and a maximum of 0.964.

Conclusions:

• The developed corpus is a valuable resource for advancing de-identification research.
• The rigorous annotation process ensured high-quality de-identification.
• Facilitated advancements in natural language processing for clinical data privacy.