An Out-of-Sample Extension to Manifold Learning via Meta-Modelling

Unsupervised manifold learning has become accepted as an important tool for reducing dimensionality of a dataset by finding its meaningful low-dimensional representation lying on an unknown nonlinear subspace. Most manifold learning methods only embed an existing dataset, but do not provide an explicit mapping function for novel out-of-sample data, thereby potentially resulting in an ineffective tool for classification purposes, particularly for iterative methods such as active learning. To address this issue, out-of-sample extension methods have been introduced to generalize an existing embedding of new samples. In this work, a novel out-of-sample method is introduced by utilizing High Dimensional Model Representation (HDMR) as a nonlinear multivariate regression with the Tikhonov regularizer for unsupervised manifold learning algorithms. The proposed method was extensively analyzed using illustrative datasets sampled from known manifolds. Several experiments with 3D synthetic datasets and face recognition datasets were also conducted, and the performance of the proposed method was compared to several well-known out-of-sample methods. The results obtained with Locally Linear Embedding (LLE), Laplacian Eigenmaps (LE), and t-Distributed Stochastic Neighbor Embedding (t-SNE) showed that the proposed method achieves competitive even better performance than the other out-of-sample methods.