F-DATA: A Fugaku Workload Dataset for Job-centric Predictive Modelling in HPC Systems.

Area of Science:

• High Performance Computing (HPC)
• Sustainable Computing
• Data Science

Background:

• High Performance Computing (HPC) systems accelerate scientific discovery but have significant environmental impacts (carbon footprint, energy consumption).
• Optimizing HPC system throughput while minimizing environmental impact requires accurate prediction of job execution characteristics.
• Developing predictive models is challenging due to the scarcity of large, public HPC datasets.

Purpose of the Study:

• To introduce F-DATA, a comprehensive public dataset of approximately 24 million jobs executed on the Fugaku supercomputer.
• To provide a rich feature set for developing and validating models that predict HPC job execution characteristics.
• To facilitate research into workload management strategies for sustainable HPC development.

Main Methods:

• Dataset compilation from ~24 million jobs executed on the Fugaku supercomputer.
• Inclusion of anonymized and irreversibly encoded sensitive job data.
• Application of a Natural Language Processing (NLP) model for encoding sensitive data while preserving predictive utility.

Main Results:

• Creation of F-DATA, a large-scale, public dataset for HPC job analysis.
• The dataset contains extensive features suitable for diverse prediction tasks.
• Sensitive job information is preserved through NLP-based encoding, addressing privacy concerns.

Conclusions:

• F-DATA addresses the critical need for public datasets in HPC research.
• The dataset supports the development of advanced workload management for efficient and sustainable supercomputing.
• Privacy-preserving data encoding enables broader use of sensitive HPC execution information.