Validation design and experimental provenance shape apparent machine-learning transferability in legacy Mössbauer...

Machine-learning models in chemistry are increasingly trained on values compiled from many published papers and are usually validated by splitting the records at random or by publication; the latter assumes that each publication is an independent experiment. We tested this assumption in a legacy ⁵⁷Fe Mössbauer corpus. All 45 source publications were re-examined against their original tables and figures, and papers reporting the same alloy composition with the same fitted spectral areas were identified and checked against the sources. Eleven publications reported measurements on three shared specimen series, so the corpus formed 37 experimental-provenance groups containing 171 model-eligible sample states. The target was an operational high-field ferromagnetic spectral fraction, not a martensite phase fraction. At a matched test-set size, median R ² fell from 0.670 under random state-level splitting to 0.322 when publications were held out and 0.287 when provenance groups were held out. With the test states held fixed, retaining related publications in training was associated with a median reduction in apparent error of 6.72 percentage points, which varied strongly between clusters and partly reflects the larger training set. The flexible chemistry-and-temperature model reached a pooled R ² of 0.422 against 0.354 for a calibrated martensite-start comparator but had the lower error in only 17 of 37 groups (group-balanced difference in mean absolute error -1.73 percentage points; 95% CI -7.87 to +4.52). Transferability improved as the evidence base expanded, although provenance-group count could not be isolated as the cause, and a retrospective open-access audit added only 12 model-eligible states. Thus, a publication identifier, which records the paper a value came from, does not guarantee experimental independence, validation design can change the apparent R ² by more than a factor of two, and a pooled advantage need not hold when each experiment is weighted equally. We provide practical recommendations for machine learning on literature-derived chemistry data and release the curated corpus, provenance assignments and code.