Discriminative structure learning of sum-product networks for data stream classification

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Sum-Product Networks (SPNs) offer exact and tractable inference for deep probabilistic models.
• Existing online SPN structure learning methods are limited to generative settings.
• Continuous data streams necessitate adaptive online learning approaches.

Purpose of the Study:

• To develop an online discriminative approach for SPN structure and parameter learning.
• To address the limitations of generative-only online SPN learning.
• To enable SPNs to adapt to time-changing class distributions in data streams.

Main Methods:

• Employing an online discriminative Sum-Product Network (SPN) learning strategy.
• Maintaining informative and representative data examples to track class distribution shifts.
• Utilizing goodness-of-fit estimation and recursive sub-SPN generation.
• Applying an outlier-robust margin-based log-likelihood loss and MPE inference for parameter updates.

Main Results:

• The proposed method effectively learns both SPN structure and parameters in an online, discriminative manner.
• Achieved superior prediction performance compared to state-of-the-art online SPN structure learners.
• Demonstrated significant speedup (order of magnitude) in processing data streams.
• Outperformed existing stream classifiers in empirical comparisons.

Conclusions:

• The developed online discriminative SPN approach is effective for learning from continuous data streams.
• This method enhances SPN capabilities by enabling discriminative learning and adaptation to evolving data.
• Offers a computationally efficient and high-performing solution for online classification tasks.