Novel Multitask Conditional Neural-Network Surrogate Models for Expensive Optimization.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Science

Background:

• Multitask learning improves model performance by leveraging shared information across related tasks, avoiding isolated learning.
• Conditional Neural Processes (CNPs) offer a framework for learning from data, but their application in multitask learning requires specialized architectures.
• Existing multitask learning models may not efficiently capture complex inter-task correlations.

Purpose of the Study:

• To propose novel multitask learning network models based on Conditional Neural Processes (CNPs).
• To introduce an extensible correlation learning layer for improved inter-task dependency modeling.
• To apply these multitask CNP networks as surrogate models within a Bayesian optimization framework.

Main Methods:

• Developed two multitask CNP (MTCNP) network architectures: one-to-many (OMc-MTCNP) and many-to-many (MMc-MTCNP).
• Integrated an extensible correlation learning layer to explicitly model task relationships.
• Utilized MTCNP networks as surrogate models in Bayesian optimization, replacing Gaussian Processes (GPs) to simplify computation.

Main Results:

• The proposed MTCNP networks effectively learn correlations among multiple tasks.
• The Bayesian optimization framework using MTCNP demonstrated competitive performance against GP-based and single-task methods.
• Augmenting datasets with related tasks using MTCNP improved model parameter estimation confidence.

Conclusions:

• The proposed OMc-MTCNP and MMc-MTCNP models offer an effective approach to multitask learning.
• MTCNP-based Bayesian optimization provides a computationally efficient and performant alternative to GP-based methods.
• Sharing knowledge across related tasks via MTCNP significantly enhances learning and optimization outcomes.