Distributed Topological Data Analysis of Tetrahedral Meshes with Apache Spark

Area of Science:

• Scientific Computing
• Computational Geometry
• Data Science

Background:

• Tetrahedral meshes are crucial for modeling complex geometries in scientific computing.
• Large-scale mesh analysis faces memory and performance challenges due to irregular data structures.
• Existing distributed solutions (e.g., MPI) often require extensive algorithm redesign.

Purpose of the Study:

• Introduce Tetra-Spark, a novel framework for distributed representation and analysis of large tetrahedral meshes.
• Leverage Apache Spark's capabilities to simplify scalable mesh processing.
• Evaluate Tetra-Spark's performance against existing Spark and MPI implementations.

Main Methods:

• Developed Tetra-Spark based on Apache Spark's data parallelism and in-memory execution.
• Implemented scalable mesh encoding, connectivity computation, and topological analysis.
• Conducted comparative experiments with vanilla Spark and MPI-based solutions.

Main Results:

• Tetra-Spark achieved up to a 78x speedup and 80% memory reduction in connectivity retrieval.
• Subsequent morphology computations showed up to a 55.6x speedup.
• Extraction of global topological descriptors was accelerated by 15.2x to 173.6x.

Conclusions:

• Tetra-Spark provides a highly effective and scalable solution for distributed tetrahedral mesh analysis.
• The framework simplifies complex computations, offering significant performance gains over traditional methods.
• Tetra-Spark demonstrates the potential of leveraging big data technologies for scientific mesh processing.