Unsupervised Discovery of Subspace Trends.

Area of Science:

• Data Science
• Machine Learning
• Bioinformatics

Background:

• High-dimensional datasets often contain hidden subspace trends obscured by noise and irrelevant features.
• Conventional dimension reduction techniques struggle to reveal these trends due to data complexity.

Purpose of the Study:

• To develop unsupervised algorithms for discovering previously unknown subspace trends in high-dimensional data.
• To overcome limitations of existing methods in handling noise, outliers, and mixed trends.

Main Methods:

• Proposed a novel graph-theoretic neighborhood similarity measure for detecting progressive changes across dimensions.
• Developed an unsupervised algorithm for trend-relevant feature selection, subspace trend discovery, and validation.
• Applied the method to synthetic and real-world biomedical datasets.

Main Results:

• Successfully identified verifiable subspace trends in diverse datasets.
• Visualizations using selected features revealed biologically meaningful hidden trends obscured by noise.
• Demonstrated the algorithm's effectiveness in uncovering complex patterns.

Conclusions:

• The proposed unsupervised approach effectively discovers and visualizes subspace trends in high-dimensional data.
• The method is broadly applicable beyond bioinformatics for exploratory data analysis, hypothesis generation, and prediction.