Cloud-based real-time enhancement for disease prediction using Confluent Cloud, Apache Kafka, feature optimization,...

Area of Science:

• Healthcare Technology
• Artificial Intelligence in Medicine
• Big Data Analytics

Background:

• Internet of Things (IoT) is generating vast healthcare data.
• Predictive real-time systems require advanced data analysis.
• Early detection of chronic kidney disease (CKD) improves patient outcomes.

Purpose of the Study:

• To develop a real-time system for early CKD detection and treatment.
• To integrate ensemble models, Explainable AI (XAI), and feature selection (FS) for predictive healthcare.
• To leverage big data streaming platforms for real-time health monitoring.

Main Methods:

• A two-phase approach was used, involving stacking models and feature selection (Genetic Algorithm - GA, Particle Swarm Optimization - PSO).
• Explainable AI (XAI) was applied to the best performing model.
• A real-time streaming pipeline was built using Confluent Cloud and Apache Kafka with Python scripts.

Main Results:

• The stacking model with GA-selected features achieved 100% accuracy, precision, recall, and F1-score in phase one.
• The real-time pipeline demonstrated the stacking model's effectiveness with 100% accuracy for CKD prediction.
• The system successfully processed streaming health data for real-time analysis.

Conclusions:

• The developed real-time system effectively detects CKD early using integrated AI and big data technologies.
• The combination of stacking models, GA for feature selection, and XAI provides a robust solution for predictive healthcare.
• Confluent Cloud and Apache Kafka enable efficient real-time data streaming for healthcare applications.