HPCGCN: A Predictive Framework on High Performance Computing Cluster Log Data Using Graph Convolutional Networks.

Area of Science:

• Computer Science
• Data Mining
• Machine Learning

Background:

• High-performance computing (HPC) environments generate vast amounts of user and task data.
• Predictive data mining in HPC is crucial for optimizing resource allocation and job scheduling.
• Existing methods often struggle to integrate diverse data sources effectively.

Purpose of the Study:

• To develop a novel Graph Convolutional Network (GCN) framework for predictive data mining in HPC.
• To integrate Slurm workload manager logs with user experience survey data.
• To predict job completion/failure and estimate CPU/memory requirements.

Main Methods:

• Constructed a heterogeneous unweighted HPC graph with multiple node types.
• Utilized a GCN for semi-supervised classification and regression tasks.
• Employed graph clustering for data partitioning.

Main Results:

• The GCN framework demonstrated significant improvements in predicting job outcomes.
• Accurate predictions for memory and CPU requirements were achieved.
• Evaluated performance using metrics like test score, F1-score, precision, recall, and R1 score against baselines.

Conclusions:

• The proposed GCN-based framework offers a powerful new approach for predictive data mining in HPC.
• Integrating diverse data sources enhances the accuracy of predictive models.
• This method can optimize HPC resource management and user experience.