Active learning strategies for the deduplication of electronic patient data using classification trees

Area of Science:

• Computer Science
• Bioinformatics
• Data Science

Background:

• Supervised record linkage demands extensive clerical review for training data.
• Active learning minimizes review costs by strategically selecting data for labeling.
• This study evaluates active learning strategies for efficient record linkage training data acquisition.

Purpose of the Study:

• To compare the efficacy of a simple active learning strategy (binary comparison patterns) against a sophisticated one (string metrics) for record linkage.
• To determine the minimal training set size required for high accuracy in record linkage.
• To assess the impact of the number of attributes on the performance of different active learning strategies.

Main Methods:

• Employed active learning on medical registry data with varying attributes to generate training sets for classification trees.
• Implemented active learning using binary comparison patterns, sampling from distinct comparison patterns.
• Utilized an extended active learning strategy incorporating Levenshtein string metric values for iterative, informative example selection.

Main Results:

• Active learning with binary comparison patterns yielded optimal results on the original dataset.
• When attributes were reduced, active learning employing string metrics demonstrated superior performance.
• Both strategies achieved high accuracy with a maximum of 200 manually reviewed training examples.

Conclusions:

• For record linkage with limited attributes (forename, name, birthday), the sophisticated active learning strategy using string metrics is recommended for high accuracy.
• The simpler active learning strategy using binary comparison patterns is suitable when more attributes are available.
• Active learning substantially decreases manual effort in training data selection for record linkage.