A non-linear optimization based robust attribute weighting model for the two-class classification problems.

Area of Science:

• Machine Learning
• Data Science
• Optimization Algorithms

Background:

• Classification performance is often limited by attribute weighting.
• Existing methods may not be sensitive to dataset structures.
• Optimizing attribute weights can enhance machine learning model performance.

Purpose of the Study:

• To develop a meta-heuristic optimization approach for determining attribute weights.
• To reduce in-class distances and increase between-class distances.
• To improve overall classification performance.

Main Methods:

• A novel mathematical model was developed as a fitness function for optimization algorithms.
• Meta-heuristic algorithms including Particle Swarm Optimization (PSO), Bat Algorithm (BAT), Gravitational Search Algorithm (GSA), and Flower Pollination Algorithm (FPA) were employed.
• Attribute weights were optimized to enhance data separation.

Main Results:

• The proposed weighting method consistently improved classification performance across all tested datasets.
• Achieved 100% accuracy on the Iris and Liver Disorders datasets.
• Significantly boosted accuracy on synthetic datasets, e.g., from 66.9% to 96.4% (Full Chain) and 64.6% to 80.2% (Two Spirals).

Conclusions:

• The meta-heuristic optimization approach effectively determines attribute weights for improved classification.
• The method enhances the performance of classifiers like Support Vector Machines (SVM) and Linear Discriminant Analysis (LDA), particularly for non-linear problems.
• Achieving 100% accuracy demonstrates the method's potential for robust data classification.