FlowCron - Increasing access to HPC by wrapping Globus into a function-as-a-service.

Area of Science:

• Computational Biology
• Scientific Computing
• Cryogenic Electron Microscopy (cryo-EM)

Background:

• Many researchers cannot leverage High-Performance Computing (HPC) due to complex setup and maintenance.
• This limits scientific throughput and timely data analysis, particularly in computationally intensive fields like cryo-EM.
• Streamlining HPC access is crucial for improving scientific quality and return on investment.

Purpose of the Study:

• To introduce FlowCron, an accessible technique for researchers to analyze data on HPC facilities.
• To reduce barriers to entry for HPC utilization, enabling faster scientific discovery.
• To enhance the reliability and ease of use of HPC data processing pipelines.

Main Methods:

• FlowCron transfers pipeline installation and upkeep from users to HPC system administrators.
• The system links Globus and cron, creating an open-source solution requiring minimal administrative support.
• Users initiate analysis via the Globus website, with automated data encryption, upload, analysis, and return.

Main Results:

• FlowCron significantly increases accessibility to HPC resources for researchers.
• The system simplifies the setup and enhances the reliability of HPC data analysis pipelines.
• Common software dependencies across HPC clusters ensure broad applicability.

Conclusions:

• FlowCron empowers researchers to utilize HPC for accelerated data analysis with minimal training.
• This approach enhances scientific throughput and improves the quality of research in data-intensive fields.
• The open-source, low-administration system offers a practical solution for democratizing HPC access.