FunDa: scalable serverless data analytics and in situ query processing

Area of Science:

• Computer Science
• Cloud Computing
• Data Analytics

Background:

• Serverless cloud computing offers a pay-what-you-use model beneficial for ephemeral tasks.
• Existing serverless paradigms struggle with stateful, long-running data analytics and query processing.

Purpose of the Study:

• To propose FunDa, an on-premises serverless data analytics framework.
• To extend the capabilities of serverless computing for complex data workloads.
• To enable serverless operations in on-premises and public cloud environments.

Main Methods:

• Extending the DaskDB system for unified data analytics and in situ SQL query processing.
• Developing a robust architecture for on-premises serverless data analytics.
• Evaluating the system on benchmarks with varying scale factors in on-premises and AWS Cloud settings.

Main Results:

• FunDa demonstrates the ability to support automatic scaling for data analytics workloads.
• The framework achieves low-latency execution of complex data analytics tasks.
• Experimental results show increased flexibility for serverless users in on-premises and cloud environments.

Conclusions:

• FunDa effectively addresses the limitations of traditional serverless computing for stateful, long-running data analytics.
• The framework provides a viable serverless solution for on-premises data processing.
• FunDa enhances the flexibility and performance of serverless data analytics in diverse environments.