REEF: Retainable Evaluator Execution Framework.
Markus Weimer1, Yingda Chen1, Byung-Gon Chun2
1Microsoft.
Proceedings. ACM-SIGMOD International Conference on Management of Data
|January 29, 2016
Summary
Resource Managers like Apache YARN offer low-level abstractions, increasing developer effort. REEF (Resource Execution and Event Framework) simplifies cloud application development by providing a control plane for task scheduling and resource coordination.
Area of Science:
- Cloud Computing
- Distributed Systems
Background:
- Resource Managers (e.g., Apache YARN) are crucial in cloud computing.
- Current abstractions for leasing cluster resources are low-level, demanding significant developer effort.
- Developers often re-implement common mechanisms like fault-tolerance and caching.
Purpose of the Study:
- Introduce REEF (Resource Execution and Event Framework) as a development framework.
- Simplify the development of elastic data processing workflows on cloud platforms.
- Provide mechanisms for resource reuse and state management.
Main Methods:
- REEF offers a control plane for scheduling and coordinating task-level work.
- It leverages cluster resources obtained from a Resource Manager.
- Provides abstractions for data caching and state management.
Main Results:
- REEF facilitates resource reuse for data caching.
- Enables easier development of elastic data processing workflows.
- Demonstrated through development of a distributed shell, ML algorithm, and CORFU system port.
Conclusions:
- REEF significantly eases the development of complex cloud applications.
- It is being used in commercial offerings like Azure Stream Analytics.
- REEF is an Apache Incubator project with broad institutional contribution.
More Related Videos
13:35Reefshape: A System for the Efficient Collection and Automated Processing of Time-Series Underwater Photogrammetry Data for Benthic Habitat Monitoring
Published on: June 13, 2025
1.8K
05:28Monitoring On-Target Signaling Responses in Larval Zebrafish - Z-REX Unmasks Precise Mechanisms of Electrophilic Drugs and Metabolites
Published on: June 2, 2023
1.5K
Related Concept Videos
01:29Self-Evaluation Maintenance Model
389
The Self-Evaluation Maintenance (SEM) model offers a psychological framework to understand how individuals’ self-esteem is influenced by the achievements of others, particularly those with whom they share close personal bonds. The SEM model operates when personal rather than social identity guides individuals. Central to this model is the notion that individuals have an inherent desire to preserve a favorable self-image, which is continuously shaped by interpersonal comparisons and...
389
01:07Elaborative Rehearsals
500
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
500
01:14Reversible and Irreversible Processes
6.2K
The thermodynamic processes can be classified into reversible and irreversible processes. The processes that can be restored to their initial state are called reversible processes. It is only possible if the process is in quasi-static equilibrium, i.e., it takes place in infinitesimally small steps, and the system remains at equilibrium However, these are ideal processes and do not occur naturally. An ideal system undergoing a reversible process is always in thermodynamic equilibrium within...
6.2K
01:34Woodward–Hoffmann Selection Rules and Microscopic Reversibility
4.2K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
4.2K
01:29Protein Folding Quality Check in the RER
5.5K
ER is the primary site for the maturation and folding of soluble and transmembrane secretory proteins. The calnexin cycle is a specific chaperone system that folds and assesses the confirmation of N-glycosylated proteins before they can exit the ER lumen. The primary players of this quality check pipeline are the lectins, ER-resident chaperones, and a glucosyl transferase enzyme. In case the calnexin system in the lumen fails to salvage a misfolded protein, it is transported to the cytoplasm...
5.5K
01:17Wald-Wolfowitz Runs Test II
641
The Wald-Wolfowitz runs test, commonly referred to as the runs test, is a nonparametric test used to assess the randomness of ordered data. The test evaluates the number of runs, which are consecutive sequences of similar elements within the data. If the number of runs is significantly higher or lower than expected, the data is considered non-random, indicating a detectable pattern or structure.
For binary data, runs are identified using symbols such as + and −, or equivalently, 1s and 0s. In...
For binary data, runs are identified using symbols such as + and −, or equivalently, 1s and 0s. In...
641