On scientific foundation models: Rigorous definitions, key applications, and a comprehensive survey

Area of Science:

• Computational Science and Engineering
• Artificial Intelligence in Science

Background:

• Traditional numerical solvers require problem-specific tuning.
• Scientific Foundation Models (SciFMs) leverage deep learning for generalizable scientific problem-solving.
• Existing SciFMs lack rigorous definitions and standardized benchmarks.

Purpose of the Study:

• To provide rigorous definitions for SciFMs based on their capabilities: domain adaptation, domain generalization, problem adaptation, and problem generalization.
• To introduce necessary conditions for operationalizing SciFM capabilities.
• To propose a taxonomy of SciFMs based on architecture, learning objectives, and training strategies.

Main Methods:

• Categorizing SciFM capabilities into four key dimensions.
• Defining sufficient and necessary conditions for SciFMs.
• Developing a taxonomy based on model architecture, learning objectives, and training strategies.
• Surveying existing SciFMs across diverse scientific domains.

Main Results:

• Rigorous definitions and a taxonomy for SciFMs are proposed.
• SciFMs are surveyed across domains like chemistry, biology, and climate science.
• Challenges such as physical consistency, interpretability, and computational demands are identified.

Conclusions:

• SciFMs hold significant potential for accelerating scientific discovery and impact.
• Addressing challenges in physical consistency, interpretability, and computational efficiency is crucial.
• Future research should focus on integrating domain knowledge and developing efficient architectures for enhanced generalization.