Improving record linkage performance in the presence of missing linkage data

Area of Science:

• Data Science
• Bioinformatics
• Health Informatics

Background:

• Traditional record linkage methods struggle with missing data, impacting efficiency and accuracy.
• Effective handling of missing linking field values is crucial for large-scale data integration.

Purpose of the Study:

• To investigate three novel methods for improving record linkage accuracy and efficiency when linking fields have missing values.
• To address the limitations of existing record linkage techniques in handling incomplete datasets.

Main Methods:

• Developed three new record linkage methods: Weight Redistribution, Distance Imputation, and Linkage Expansion, by extending Fellegi-Sunter scoring in the FRIL software.
• Weight Redistribution adjusts weights of available fields; Distance Imputation imputes distances between missing values; Linkage Expansion incorporates additional fields.
• Methods were tested using simulated datasets with varying rates of field value corruption.

Main Results:

• The novel methods demonstrated high sensitivity (.895–.992) and positive predictive values (PPV) (.865–1) in datasets with low corruption rates.
• Sensitivity decreased across all methods as data corruption rates increased.
• The developed algorithms show promise for accurate and efficient record linkage.

Conclusions:

• The new record linkage algorithms offer a promising solution for efficiently combining large patient-level datasets.
• These methods can significantly support biomedical and clinical research by improving data integration accuracy.
• Further validation and application in real-world scenarios are warranted.