Dual-level clustering ensemble algorithm with three consensus strategies.

Area of Science:

• Data Science
• Machine Learning
• Artificial Intelligence

Background:

• Clustering ensemble (CE) methods are powerful for data analysis but suffer from issues with parameter dependency, one-sided co-association matrix construction, and limited conflict reconciliation.
• Existing CE selection strategies often lack adaptivity and rely on empirical knowledge, hindering optimal performance.

Purpose of the Study:

• To propose a novel dual-level clustering ensemble algorithm addressing key limitations in current CE methods.
• To enhance the adaptive selectivity, co-association matrix construction, and conflict reconciliation capabilities within clustering ensembles.

Main Methods:

• A backward clustering ensemble selection framework with adaptive member elimination.
• Two modified relation matrices incorporating spatial location and co-occurrence frequency for base clustering consensus.
• An adjustable Dempster-Shafer evidence theory for dynamic fusion of multiple ensemble results at the CE consensus level.

Main Results:

• The proposed dual-level CE algorithm demonstrates superior consensus ability compared to seven state-of-the-art and typical CE algorithms.
• Experimental results validate the algorithm's enhanced robustness and performance in clustering tasks.

Conclusions:

• The developed dual-level clustering ensemble algorithm effectively overcomes existing challenges in CE methods.
• The integration of adaptive selection, improved co-association, and Dempster-Shafer theory leads to significant improvements in consensus and robustness.