Missing values in deduplication of electronic patient data.

Area of Science:

• Data Science
• Computer Science
• Statistics

Background:

• Systematic methods for handling missing values in record linkage are underdeveloped.
• This study evaluates an ad-hoc approach of treating unknown comparison values as 'unequal' against other imputation and classification strategies.
• Empirical evaluations were performed on both real-world and simulated datasets with missing values.

Purpose of the Study:

• To compare the effectiveness of different missing value handling strategies in record linkage.
• To evaluate the performance of an ad-hoc 'unequal' imputation method.
• To determine the optimal approach for dealing with missing data in deduplication tasks.

Main Methods:

• Utilized cancer registry and artificial datasets with introduced missing values.
• Employed classification and regression trees for analysis.
• Compared imputation with unique values (zero or 0.5), sample-based imputation, reduced-model classification, and complete-case induction.
• Evaluated methods based on training data requirements and F-scores.

Main Results:

• Imputation with a unique value, specifically zero, yielded the best overall results.
• Imputation with zero requires significantly less training data and simplifies computation by preserving binary data structure.
• While imputation with 0.5 showed higher median F-scores, imputation with zero offered a better balance of performance and efficiency.

Conclusions:

• The findings support the ad-hoc strategy of replacing missing values with an 'unequal' indicator for record linkage.
• This conclusion is drawn from evaluations on specific deduplication methods and datasets.
• Further empirical analyses and applications are recommended to confirm these results.