How Well Do Automated Linking Methods Perform? Lessons from U.S. Historical Data

Area of Science:

• Social Sciences
• Computer Science
• Statistics

Background:

• Historical record linkage is vital for social science research, enabling longitudinal and cross-sectional data analysis.
• Existing record linkage algorithms vary in their assumptions and performance, with limited understanding of their impact on data quality.

Purpose of the Study:

• To evaluate the performance of common record linkage algorithms using high-quality datasets.
• To assess the direct effects of algorithm choices and assumptions on data quality and research inferences.

Main Methods:

• Literature review of historical record linkage in the U.S.
• Performance evaluation of record linking algorithms using two hand-linked datasets and one synthetic dataset.
• Analysis of linking errors and their relationship to sample characteristics.

Main Results:

• No single algorithm, including manual linking, consistently yields representative samples.
• 15-37% of algorithm-generated links were identified as errors by human reviewers.
• False links systematically correlate with baseline sample characteristics, introducing potential measurement error.

Conclusions:

• Linking errors can significantly attenuate research findings, such as estimates of intergenerational income elasticity.
• Current trends towards automated linking and higher link rates may exacerbate the impact of errors.
• Recommendations are provided for reducing linking errors and future research directions.