Enhanced virtual sample generation based on manifold features: Applications to developing soft sensor using small data | JoVE Visualize

Area of Science:

Process Industry
Data Science
Machine Learning

Background:

Accurate prediction of key variables is crucial in process industries, often requiring data-driven soft sensors.
The performance of soft sensors is heavily dependent on data availability, posing a challenge with limited sample sizes (small sample problem).
Generating virtual samples is a viable strategy to augment small datasets and improve model performance.

Purpose of the Study:

To propose an enhanced virtual sample generation method, t-SNE-based virtual sample generation (t-SNE-VSG), for developing soft sensors with small datasets.
To leverage manifold features for more informative virtual sample creation.
To validate the effectiveness of t-SNE-VSG in improving soft sensor accuracy using both standard and real-world industrial data.

Main Methods:

Feature extraction using T-Distribution Stochastic Neighbor Embedding (t-SNE) to capture data distribution characteristics.
Generation of virtual t-SNE input features via interpolation algorithms.
Utilization of the random forest algorithm to predict virtual outputs based on virtual t-SNE input features.
Development of the t-SNE-based virtual sample generation (t-SNE-VSG) method.

Main Results:

The proposed t-SNE-VSG method effectively improved the accuracy of soft sensors developed with small datasets.
Validation on a standard dataset and a real industrial dataset (Purified Terephthalic Acid process) confirmed the method's feasibility and effectiveness.
Simulation results demonstrated that adding t-SNE-VSG generated virtual samples significantly enhanced soft sensor performance.
t-SNE-VSG achieved superior accuracy compared to state-of-the-art virtual sample generation techniques.

Conclusions:

The t-SNE-VSG method is a valuable approach for addressing the small sample problem in developing data-driven soft sensors.
The integration of manifold features through t-SNE enhances the quality and informativeness of generated virtual samples.
The proposed method offers a practical solution for improving the accuracy and reliability of soft sensors in data-scarce industrial environments.