Objective-Dependent Active Learning and Calibrated Uncertainty for Sample-Efficient Discovery of Rare-Earth...

Measuring distribution coefficients (log D) for rare-earth solvent extraction is slow and expensive, making sample-efficient machine learning attractive. However, the practical questions of which acquisition strategy to use, whether the attendant uncertainty estimates can be trusted, and how well a model actually generalizes to new chemistry are rarely answered together on real data. We benchmark active learning (AL) on 1,202 experimental log D measurements for lanthanide extraction (93 ligands, 14 lanthanides, 45 literature sources), with prospective validation on four independently synthesized ligands. Five findings emerge, each based on real data with multiseed statistics. (i) A plain random forest reaches R ² = 0.94/RMSE = 0.33 on the published row-level validation split, outperforming the previously reported deep network (R ² = 0.85). However, under leakage-controlled splits that forbid a ligand, source, or structural family from appearing in both train and test, the estimate collapses (grouped-CV R ² = 0.20 to -0.03), so the in-distribution number is strongly optimistic, and realistic deployability is far lower. (ii) The optimal AL acquisition is objective-dependent: exploitation recovers all top row-level extractants (top-10 recall = 1.0) but yields a biased, poorly predictive model, whereas exploration/random gives the best predictive model but finds few of the best. No strategy dominates both, and this survives a chemically realistic ligand-batch acquisition unit and reproduces on an independent 4,200-compound log D benchmark. (iii) The discovery ranking of strategies inverts with the definition of a "top" system: exploitation dominates for condition-specific rows, but broad sampling already recovers most top ligands. (iv) For prediction, AL provides no advantage over random selection. (v) Conformal calibration is essential for trustworthy uncertainty (an 8-fold in-distribution error reduction) but does not improve AL acquisition, which depends only on the uncertainty ranking. Under distribution shift, global conformal leaves subgroup imbalance that group-conditional (Mondrian) calibration reduces, albeit at the cost of reordering the ranking. We compare five model/uncertainty estimators, add early recognition discovery metrics, and distill the results into a practical decision guide, releasing all code and data.