A simple two-step procedure using the Fellegi-Sunter model for frequency-based record linkage.

Area of Science:

• Data Science
• Statistics
• Computer Science

Background:

• The Fellegi-Sunter model for probabilistic record linkage does not utilize the information within field values.
• Current frequency-based methods often require prior data or training sets, limiting their applicability.
• Agreement on rare values is a stronger indicator of a match than agreement on common values.

Purpose of the Study:

• To propose a novel, simple two-step procedure for frequency-based matching within the Fellegi-Sunter framework.
• To enhance record linkage accuracy by leveraging the discriminating power of rare field value agreements.
• To develop a method that does not rely on external data or prior studies for error probabilities and value frequencies.

Main Methods:

• A two-step procedure is proposed to adjust matching weights based on frequency distributions of agreed field values.
• The Fellegi-Sunter model is used to estimate these frequency distributions from the data itself.
• The method's performance is evaluated through a real-world application and simulation studies.

Main Results:

• The proposed frequency-based matching method demonstrates comparable or superior performance to the unadjusted Fellegi-Sunter method.
• The method shows significant improvements in matching accuracy, particularly when using fields with lower discriminating power.
• The benefits of frequency-based matching diminish as the discriminating power of the matching fields increases.

Conclusions:

• The proposed method effectively incorporates field value information into probabilistic record linkage without external data.
• This approach offers a practical solution for enhancing record linkage accuracy in diverse applications.
• Frequency-based adjustments are most impactful when dealing with less informative matching fields.