Online Discriminative Kernel Density Estimator With Gaussian Kernels.

Area of Science:

• Machine Learning
• Statistical Modeling
• Pattern Recognition

Background:

• Supervised online estimation of probabilistic discriminative models is crucial for classification tasks with streaming data.
• Existing methods like online kernel density estimators (oKDE) can be computationally intensive and produce complex models.
• Maintaining model simplicity while preserving discriminative power is a key challenge in online learning.

Purpose of the Study:

• To propose a novel supervised online estimation method for probabilistic discriminative models.
• To develop a classifier that estimates class distributions using Gaussian mixture models (GMMs) from data streams.
• To introduce a compression strategy that minimizes the loss of inter-class discrimination.

Main Methods:

• The proposed method, online discriminative kernel density estimator (odKDE), utilizes an online kernel density estimator (oKDE) for distribution updates.
• GMMs are periodically compressed to maintain low model complexity.
• A new cost function is introduced to guide compression, preserving inter-class discriminative properties.

Main Results:

• The odKDE achieves classification performance comparable to state-of-the-art batch kernel density estimators (KDEs) and Support Vector Machines (SVMs).
• The method allows for online adaptation from large datasets.
• odKDE produces models with lower complexity compared to the standard oKDE.

Conclusions:

• The odKDE offers an effective solution for supervised online classification, balancing performance, adaptability, and model simplicity.
• The novel compression strategy successfully maintains discriminative power while reducing model complexity.
• odKDE represents a significant advancement for real-time classification tasks with evolving data.