Surrogate- and invariance-boosted contrastive learning for data-scarce applications in science

Area of Science:

• Computational Science
• Materials Science
• Quantum Mechanics
• Photonics

Background:

• Deep learning is increasingly used in natural sciences for tasks like property prediction and material discovery.
• Training deep learning models typically requires large amounts of labeled data, which is often scarce and expensive to obtain in scientific research.
• Auxiliary information sources are often available in scientific problems but are underutilized in current deep learning frameworks.

Purpose of the Study:

• To develop a novel deep learning framework to address data scarcity challenges in scientific applications.
• To incorporate readily available auxiliary information—unlabeled data, prior knowledge of symmetries/invariances, and low-cost surrogate data—to enhance model training.
• To demonstrate the framework's effectiveness and generalizability across diverse scientific problems.

Main Methods:

• Introduced surrogate- and invariance-boosted contrastive learning (SIB-CL), a deep learning framework.
• SIB-CL integrates three key auxiliary information sources: abundant unlabeled data, known physical symmetries or invariances, and cost-efficient surrogate data.
• The framework was applied to predict the density-of-states of 2D photonic crystals and solve the 3D time-independent Schrödinger equation.

Main Results:

• SIB-CL demonstrated significant effectiveness and generality across different scientific problems.
• The framework achieved orders of magnitude reduction in the number of required labels while maintaining high network accuracy.
• SIB-CL successfully leveraged auxiliary information to overcome data limitations inherent in scientific deep learning tasks.

Conclusions:

• SIB-CL offers a powerful solution for data-scarce deep learning in the natural sciences.
• The method significantly lowers the barrier to entry for applying advanced machine learning techniques to scientific discovery.
• SIB-CL's ability to utilize inexpensive auxiliary data makes it a practical and efficient approach for scientific research.