Spectral Learning for Supervised Topic Models.

Area of Science:

• Machine Learning
• Natural Language Processing
• Statistical Modeling

Background:

• Supervised topic models (sLDA) integrate latent topic structure with response variables.
• Current inference methods (variational, Monte Carlo) are prone to local minima.
• Spectral methods offer provable guarantees for unsupervised topic models like LDA.

Purpose of the Study:

• Investigate spectral methods for supervised LDA (sLDA) parameter recovery.
• Develop efficient and provably correct spectral algorithms for sLDA.
• Establish theoretical guarantees and assess practical performance.

Main Methods:

• A two-stage spectral method: LDA parameter recovery followed by regression parameter estimation.
• A single-phase spectral algorithm for joint recovery of topic distributions and regression weights.
• Theoretical analysis including sample complexity bounds and identifiability conditions.

Main Results:

• Developed provably correct and computationally efficient spectral algorithms for sLDA.
• Established sample complexity bounds and identifiability conditions for sLDA.
• Demonstrated superior or comparable performance to state-of-the-art methods on synthetic and real-world datasets, including large-scale review data.

Conclusions:

• Spectral methods are effective for supervised topic modeling.
• The proposed single-phase spectral algorithm offers a powerful and efficient approach to sLDA.
• This work advances the field by providing theoretically grounded and practically effective solutions for sLDA.