AdaptMol: domain adaptation for molecular image recognition with limited supervision

Optical Chemical Structure Recognition (OCSR) aims to convert two-dimensional molecular images into machine-readable formats such as SMILES strings. Deep learning has substantially improved OCSR performance, yet most methods rely on synthetic training data and struggle to generalize to real-world inputs, especially hand-drawn diagrams, where stroke width, geometry, and drawing conventions vary widely across individuals. In this work, we propose an image-to-graph model AdaptMol that enables effective transfer from synthetic to real-world data without requiring manual graph annotations in the target domains. AdaptMol is an integrated pipeline that starts with training a base model on synthetic data, and then refines model representations through unsupervised domain adaptation and self-training. Our key insight is that bond features are domain-invariant in nature; they encode structural relationships between atoms that are independent of visual variations across domains. Thus, during domain adaptation, we align bond-level feature distributions via class-conditional Maximum Mean Discrepancy (MMD) to enforce cross-domain consistency. We also design a comprehensive data augmentation strategy to enhance the robustness of the base model, facilitating stable self-training on unlabeled target samples. On hand-drawn molecular images, our model achieves 82.6% accuracy and outperforms the best prior method by 10.7 points, while maintaining competitive performance across four benchmarks comprising molecular images from scientific literature and patent documents.Scientific contributionWe propose AdaptMol, an image-to-graph model that predicts molecular structures as graphs of atoms and bonds, achieving effective transfer from synthetic to hand-drawn molecular images without requiring target domain graph annotations. We combine class-conditional Maximum Mean Discrepancy to align bond features across domains with comprehensive data augmentation to increase training data variation, jointly improving base model accuracy sufficiently for self-training and addressing the critical failure mode of prior approaches that begin with insufficient accuracy. We further introduce a dual position representation that supervises atom positions through both discrete coordinate tokens and continuous spatial heatmaps to reduce false positives in atom localization.