iQCAR: A Demonstration of an Inter-Query Contention Analyzer for Cluster Computing Frameworks.

Area of Science:

• Database Systems
• Distributed Computing
• Performance Analysis

Background:

• Shared clusters face unpredictable query runtimes due to resource contention from inter-query interactions.
• Identifying the root cause of these contentions is crucial for optimizing performance.

Purpose of the Study:

• Introduce iQCAR (inter Query Contention Analyzer), a system to formally model and diagnose inter-query resource contentions.
• Provide a framework for attributing blame and understanding the impact of contention on query performance.

Main Methods:

• Leverage a multi-level directed acyclic graph (iQC-Graph) to model interferences between concurrent queries.
• Develop an interactive interface for step-wise exploration of query execution stages and resource contention.

Main Results:

• iQCAR enables identification of top-k contention sources and victims.
• The system diagnoses high-contention nodes/resources and ranks their performance impact.
• Users can explore how scheduler rules resolve contentions in subsequent executions.

Conclusions:

• iQCAR offers a systematic approach to analyze and resolve query runtime unpredictability in shared clusters.
• The tool facilitates deep exploration and understanding of resource contention dynamics.
• Effective contention management through iQCAR can significantly improve query performance.