Hashedcubes: Simple, Low Memory, Real-Time Visual Exploration of Big Data.

Area of Science:

• Computer Science
• Data Visualization
• Database Systems

Background:

• Large datasets pose challenges for real-time visual exploration due to high memory and latency requirements.
• Existing data cube visualization methods often struggle with scalability and efficiency.

Purpose of the Study:

• To propose Hashedcubes, a new data structure designed for efficient, real-time visual exploration of large datasets.
• To demonstrate Hashedcubes' advantages in memory efficiency, query speed, and implementation simplicity compared to state-of-the-art methods.

Main Methods:

• Development of algorithms for building and querying Hashedcubes.
• Integration of Hashedcubes with interactive visualizations like binned scatterplots, linked histograms, and heatmaps.
• Empirical evaluation of memory usage, build time, and query latencies on synthetic and real-world datasets.

Main Results:

• Hashedcubes achieves significantly lower memory requirements, up to two orders of magnitude less than existing proposals.
• Query latencies are low, with typical queries answered fast enough to sustain interaction, even on datasets with hundreds of millions of elements.
• While some queries may be slightly slower than the state-of-the-art, 98% of queries complete within 40ms.

Conclusions:

• Hashedcubes provides a highly efficient solution for real-time visual exploration of large-scale data.
• Its low memory footprint and fast query performance make it suitable for interactive data analysis.
• Future work can explore spacetime tradeoffs and further optimizations for the Hashedcubes data structure.