The MITRE Identification Scrubber Toolkit: design, training, and assessment.

Area of Science:

• Health Informatics
• Medical Data Security
• Natural Language Processing

Background:

• Medical records require de-identification for data sharing, a process often manual and time-consuming.
• Existing automated de-identification software lacks generalizability across different medical document types.

Purpose of the Study:

• To evaluate the MITRE Identification Scrubber Toolkit (MIST) for automated de-identification of diverse patient record types.
• To assess MIST's capability for rapid tailoring to specific document formats using machine learning.

Main Methods:

• MIST was trained and tested on four classes of de-identified patient records: discharge summaries, laboratory reports, letters, and order summaries.
• Performance was evaluated using precision, recall, F-measure, and accuracy at the word level.
• Experiments assessed MIST's performance on individual record classes and pooled datasets.

Main Results:

• MIST achieved high de-identification performance, with F-measures ranging from 0.934 (laboratory reports) to 0.996 (order and discharge summaries).
• Performance varied based on the type and amount of protected health information (PHI) present in the records.
• The toolkit requires approximately several hundred training examples to achieve an F-measure of at least 0.9.

Conclusions:

• MIST enables rapid adaptation of automated de-identification to specific medical document types.
• The toolkit facilitates the adoption of de-identification software by medical end-users without requiring access to original patient data.
• MIST is released as open-source software to promote its use across various institutions and datasets.