Locally Embedding Autoencoders: A Semi-Supervised Manifold Learning Approach of Document Representation

Area of Science:

• Natural Language Processing
• Machine Learning
• Information Retrieval

Background:

• Topic models and neural networks are key for low-dimensional text representation.
• Existing models lack discriminative power, as representations depend on the entire corpus.
• This limits their ability to provide distinct document representations.

Purpose of the Study:

• To develop a semi-supervised manifold-inspired autoencoder for extracting discriminative document representations.
• To address the limitation of non-discriminatory latent representations in current models.
• To improve the quality of latent representations for downstream tasks like clustering and classification.

Main Methods:

• A semi-supervised manifold-inspired autoencoder is proposed.
• Discriminative neighbors are identified using Euclidean distance in observation spaces.
• The autoencoder is trained by minimizing reconstruction error and neighbor representation error.

Main Results:

• The proposed method achieved over 15% improvement in document clustering.
• A nearly 7% improvement was observed in document classification tasks.
• The model effectively captures discriminative latent representations for new documents.

Conclusions:

• The manifold-inspired autoencoder successfully extracts more discriminative latent document representations.
• The method enhances performance in document clustering and classification.
• It offers a way to discover meaningful word combinations and improve representation comprehensibility.