Comparing optimal transport and machine learning approaches for databases merging in scenarios involving missing data...

One of the challenges encountered when merging heterogeneous observational medical datasets is the recoding of categorical target variables that may have been measured differently across data sources. This study compares standard machine learning-based approaches, specifically Multiple Imputation by Chained Equations (MICE), k-Nearest Neighbours (kNN), missForest, and Factor Analysis of Mixed Data (missMDA), with an Optimal Transport-based algorithm (OTrecod). Their empirical performance is evaluated in realistic data integration settings that incorporate missing covariate values, non-linear relationships, and imbalanced groups, all of which remain underexplored.

A comprehensive simulation study was conducted, varying sample size, group imbalance, signal-to-noise ratio, non-linear constraints, and missing data mechanisms and proportions. The results reveal two distinct performance tiers, with missForest, missMDA, and OTrecod often outperforming MICE and kNN. While missForest achieves the highest recoding accuracy at low missingness levels, OTrecod and missMDA show superior robustness in high missingness scenarios. Furthermore, OTrecod excels under severe non-linear constraints. These findings are further supported by subsets of the National Child Development Study, in which OTrecod produced the most stable and consistent recoding alignments across methods.

The source code supporting this study is publicly available at https://github.com/FloAI/CompareOT and archived on Zenodo (DOI: 10.5281/zenodo.20542318).