Related Experiment Video
Updated: Jul 10, 2026

Databases to Efficiently Manage Medium Sized, Low Velocity, Multidimensional Data in Tissue Engineering
Published on: November 22, 2019
Optimizing Interactive Development of Data-Intensive Applications
Matteo Interlandi1, Sai Deep Tetali1, Muhammad Ali Gulzar1
1University of California, Los Angeles.
Abstract:
Modern Data-Intensive Scalable Computing (DISC) systems are designed to process data through batch jobs that execute programs (e.g., queries) compiled from a high-level language. These programs are often developed interactively by posing ad-hoc queries over the base data until a desired result is generated. We observe that there can be significant overlap in the structure of these queries used to derive the final program. Yet, each successive execution of a slightly modified query is performed anew, which can significantly increase the development cycle. Vega is an Apache Spark framework that we have implemented for optimizing a series of similar Spark programs, likely originating from a development or exploratory data analysis session. Spark developers (e.g., data scientists) can leverage Vega to significantly reduce the amount of time it takes to re-execute a modified Spark program, reducing the overall time to market for their Big Data applications.
Related Concept Videos
Production Efficiency
Column Efficiency: Rate Theory
During elution, a solute molecule experiences numerous transitions between stationary and mobile phases, exhibiting irregular residence times in...
Parallel Processing
Optimization Problems
Methods of Medium Optimization
Scale-Up Processes

