Interpretable Active Learning for Pedigree Data Deduplication in Cancer Genetics

Studying rare genetic conditions often requires multicenter research to gather sufficient data. However, data from multiple institutions may include relatives from the same family enrolled at different sites, increasing the likelihood of duplicate records. This issue is compounded by the use of deidentified data, which limits the direct linkage through personal identifiers. These redundancies can bias family-based genetic studies underscoring the need for robust methods for pedigree deduplication. We propose an interpretable, active learning-based approach to efficiently identify duplicate records in genetic studies, with specific application to families with TP53 mutations in the Li-Fraumeni and TP53: Understanding and Progress (LiFT UP) study.

Our approach combines heuristic labeling with graph-based features and a machine learning model to iteratively refine duplicate detection. We first generate a partially labeled data set leveraging mutation variant diversity and family characteristics. A random forest classifier is then trained to predict duplicate pairs, with active learning guiding iterative refinement. This method is applied to real-world pedigree data from the LiFT UP study to assess its effectiveness in a multicenter setting.

Our method labeled pedigree pairs in data from the LiFT UP study with a high degree of automation, achieving 99.95% automated processing in the deduplication workflow. By prioritizing likely duplicates for human review, it minimized manual effort while aiming for high specificity. This automated approach avoids dependence on rule-based filters, such as identifier matching, which ultimately require manual confirmation, offering a more scalable solution for improving data quality in risk estimation.

Interpretable active learning provides an effective solution for pedigree deduplication. Future work will explore refinements in identifying potential duplicates and evaluate its generalizability across other genetic data sets.