Ensemble manifold regularization.

Area of Science:

• Machine Learning
• Data Science
• Computer Science

Background:

• Semi-supervised learning (SSL) requires effective intrinsic manifold approximation.
• Optimizing hyperparameters for manifold learning is challenging, often relying on inefficient cross-validation or grid search.
• Existing methods can suffer from suboptimality and overfitting.

Purpose of the Study:

• To propose an automatic and scalable framework for approximating the intrinsic manifold in general SSL problems.
• To address the challenges of hyperparameter optimization and potential suboptimality in manifold learning.
• To develop a method that jointly learns the manifold and the SSL model.

Main Methods:

• Developed an Ensemble Manifold Regularization (EMR) framework.
• EMR combines multiple initial manifold guesses to approximate the intrinsic manifold.
• The framework jointly learns the composite manifold and the semi-supervised learner.
• EMR implicitly optimizes hyperparameters and is designed for scalability.

Main Results:

• EMR is fully automatic in learning intrinsic manifold hyperparameters.
• The framework demonstrates conditional optimality for intrinsic manifold approximation.
• EMR is scalable in time and space for numerous candidate hyperparameters.
• Convergence to a deterministic matrix at a root-n rate is proven.
• Experiments on synthetic and real datasets show EMR's effectiveness.

Conclusions:

• The proposed EMR framework offers an effective, automatic, and scalable solution for intrinsic manifold approximation in SSL.
• EMR overcomes limitations of traditional hyperparameter optimization methods.
• The framework shows strong performance across diverse datasets.