A novel ensemble-based genetic algorithm for enhanced feature selection in lung and pancreatic cancer diagnosis

A major challenge encountered in high dimensional cancer datasets is the presence of irrelevant or redundant features, which can decrease the performance of the classifier model. An efficient feature selection technique is required in order to develop accurate and clinically relevant models for prediction or early detection of cancers. So, in order to enhance feature selection and improve diagnostic accuracy, especially in cancer datasets, this study introduces a novel framework, i.e., Best Cost Genetic Algorithm with Ensemble Crossover Operation (BCGA-ECO). The BCGA-ECO framework integrates three complementary crossover techniques, i.e., Single Point Crossover (SPC), Double Point Crossover (DPC), and Uniform Arithmetic Crossover (UAC), and evaluates chromosomes based on a composite objective function combining mean squared error (MSE) and feature compactness. Experiments were conducted using three datasets: the UCI Lung Cancer dataset containing 32 samples with 56 features, the GEO pancreatic gene expression dataset (GDS4100) containing 24 samples with 76 features, and a clinical pancreatic tumor dataset comprising 74 patient records with 20 clinical features. The performance was assessed using accuracy, RMSE, F1-score, and AUC-ROC. Comparative benchmarks included BestFit, GANN, ReliefF, LASSO, mRMR, etc. BCGA-ECO has shown better performance on these datasets. For the lung cancer dataset (from UCI Repository), BCGA-ECO achieved 93.5% accuracy with an RMSE of 0.209. Therefore, BCGA-ECO outperformed individual crossover methods and feature selectors. For the pancreatic dataset (GDS4100), BCGA-ECO achieved 95.6% accuracy with an RMSE of 0.182. This method helped improve convergence, feature diversity, and robustness. The improvements were consistent across datasets. Additionally, a clinical dataset from pancreatic tumor patients was used to predict post-surgical patient survival. The proposed BCGA-ECO achieved improved efficiency (MSE = 0.079) compared with the traditional GA-NN model (MSE = 0.087), further demonstrating its robustness for moderate- to high-dimensional clinical datasets (typically tens to a few hundred features), achieving AUC-ROC values of 0.962 for lung cancer and 0.974 for pancreatic cancer datasets. Also, ablation studies verified that BCGA-ECO outperformed its single crossover counterparts (SPC, DPC, and UAC).