Deep siamese residual support vector machine with applications to disease prediction

Support Vector Machines (SVMs) excel in classification and regression tasks involving high-dimensional nonlinear data, boasting high accuracy, strong generalization ability, and robust performance. Particularly noteworthy is their outstanding performance when integrated into end-to-end collaborative frameworks with deep learning models. However, such frameworks often leverage the strengths of deep learning and SVMs separately. To achieve a synergistic effect that surpasses the mere sum of its parts, this paper proposes an end-to-end learning model called the Deep Siamese Residual Support Vector Machine (DSRSVM), which integrates deep neural networks and SVMs. The learning process of the model consists of two phases: deep residual network siamese pre-training and deep residual support vector machine fine-tuning. During the deep siamese pre-training phase, the model leverages the deep residual network to capture similarities and differences in data features. Subsequently, an SVM is embedded within the pre-trained deep residual network to construct the DSRSVM model. The SVM loss is then propagated backward to the deep residual neural network using a gradient descent algorithm, enabling end-to-end learning of the DSRSVM. This paper presents the DSRSVM training algorithm and provides a theoretical proof of its convergence. The algorithm was validated on publicly available medical datasets, demonstrating superior performance in prediction accuracy, recall, and F1 score compared to traditional end-to-end collaborative frameworks of deep learning and SVMs. These results affirm that the DSRSVM model achieves a synergistic improvement, exemplifying the principle of "Synergy creates greater outcomes".