Heavy-tailed kernels reveal a finer cluster structure in t-SNE visualisations

Area of Science:

• Computational statistics
• Data visualization
• Machine learning

Background:

• T-distributed stochastic neighbour embedding (t-SNE) is a popular technique for visualizing high-dimensional data.
• Standard t-SNE uses a heavy-tailed Cauchy kernel to address the crowding problem inherent in its predecessor, SNE.
• However, the fixed kernel in t-SNE may limit its ability to reveal all underlying cluster structures.

Purpose of the Study:

• To develop an efficient t-SNE implementation with a tunable t-distribution kernel.
• To investigate the impact of varying the kernel's degree of freedom (ν) on data visualization.
• To explore the potential of modified t-SNE kernels for uncovering finer cluster details.

Main Methods:

• Implementation of t-SNE with a generalized t-distribution kernel, parameterized by the degree of freedom ν.
• Theoretical analysis and application to toy datasets to understand the effect of ν.
• Demonstration on large, real-world datasets including MNIST, single-cell RNA-sequencing, and HathiTrust library data.

Main Results:

• The generalized t-SNE efficiently visualizes high-dimensional data using a tunable kernel.
• Values of ν < 1 further mitigate the crowding problem and reveal finer cluster structures.
• Meaningful clusters, invisible with standard t-SNE, were identified in large datasets.

Conclusions:

• Modifying the tail heaviness of the t-SNE kernel offers enhanced data exploration capabilities.
• The generalized t-SNE provides a more flexible approach to uncovering complex cluster patterns.
• This method yields additional insights into the intrinsic structure of diverse datasets.