Learning via variably scaled kernels

Area of Science:

• Machine Learning
• Computational Biology
• Data Science

Background:

• Standard kernels in machine learning can have limitations in expressiveness and stability.
• Support Vector Machines (SVMs) and Kernel Regression Networks (KRNs) are widely used learning models.
• Feature extraction can be computationally intensive for classification tasks.

Purpose of the Study:

• To investigate the efficacy of variably scaled kernels (VSKs) for machine learning tasks.
• To compare VSKs with standard kernels in terms of expressiveness and stability.
• To explore VSKs as an alternative to traditional feature extraction methods.

Main Methods:

• Utilized variably scaled kernels (VSKs) in Support Vector Machine (SVM) classifiers and Kernel Regression Networks (KRNs).
• Proposed and evaluated various scaling functions for VSK implementation, including a probabilistic approach for SVMs using the Naive Bayes (NB) classifier.
• Conducted numerical experiments and applied the methods to real-world datasets (breast cancer, COVID-19).

Main Results:

• Variably scaled kernels (VSKs) demonstrated superior expressiveness and stability compared to standard kernels under specific assumptions.
• The proposed scaling functions, including the Naive Bayes-based approach for SVMs, proved effective for VSK implementation.
• Numerical results confirmed the advantages of VSKs on breast cancer and COVID-19 datasets.
• VSKs offer a viable, computationally efficient alternative to demanding feature extraction procedures in classification.

Conclusions:

• Variably scaled kernels (VSKs) represent a significant advancement for machine learning, enhancing model performance.
• The proposed VSK framework provides more expressive and stable models, particularly for SVMs and KRNs.
• VSKs offer practical advantages, including potential computational savings in feature extraction for classification tasks.