Transfer learning using attentions across atomic systems with graph neural networks (TAAG).

Area of Science:

• Computational chemistry
• Materials science
• Machine learning

Background:

• Graph Neural Networks (GNNs) have advanced molecular and catalyst discovery.
• Existing GNN models are often domain-specific (e.g., small molecules or materials), requiring extensive data.
• Transfer learning (TL) offers a computationally efficient way to generalize models across domains.

Purpose of the Study:

• To evaluate the effectiveness of transfer learning for GNNs in molecular and catalyst discovery.
• To investigate how pretrained GNN layers adapt to different datasets and tasks.
• To develop an improved TL approach for cross-domain generalization.

Main Methods:

• A GNN model pretrained on the Open Catalyst Dataset (OC20) was fine-tuned on MD17 and the *CO adsorbate dataset.
• Analysis of GNN layer representations to understand feature transfer.
• Development and evaluation of a novel attention-based TL method (TAAG).

Main Results:

• Pretrained GNNs show improved performance on in-domain tasks (53% for *CO, 17% for OCP tasks) and faster training (up to 4x speedup).
• Initial GNN layers capture generalizable features, while final layers are task-specific.
• The proposed TAAG method achieved a 6% mean improvement on out-of-domain datasets like MD17 compared to training from scratch.

Conclusions:

• Transfer learning is a viable strategy for GNNs in molecular and catalyst discovery, but performance varies across domains.
• The TAAG method effectively adapts GNNs for cross-domain generalization by prioritizing important features.
• This work highlights the potential of attention mechanisms to enhance transfer learning in scientific applications.