Error-Guaranteed Compression with Preservation of Downstream Quantities for Electron MicroscopyȠ

Area of Science:

• Materials Science
• Data Science
• Microscopy

Background:

• 4D Scanning Transmission Electron Microscopy (4D STEM) generates massive datasets, exceeding storage and bandwidth capabilities.
• Existing compression methods (lossless and lossy) have limitations in ratio or data integrity.
• Accurate downstream analysis of 4D STEM data is crucial for scientific discovery.

Purpose of the Study:

• To develop a compression method for 4D STEM data that preserves essential features and statistical moments.
• To define a metric for evaluating compression accuracy in diffraction space.
• To enable consistent recovery of moment-derived observables from compressed data.

Main Methods:

• A moment-preserving compression workflow combining multigrid adaptive reduction (MGARD) and constraint satisfaction.
• MGARD provides mathematically guaranteed error bounds on raw data.
• Constraint satisfaction rectifies decompressed data to preserve statistical moments (e.g., Center-of-Mass).

Main Results:

• The proposed method significantly reduces errors in targeted downstream quantities.
• Moment-derived observables are recovered consistently from the corrected tensor.
• A frequency-domain reliability criterion helps evaluate acceptable accuracy limits.

Conclusions:

• The moment-preserving compression workflow offers a practical solution for managing large 4D STEM datasets.
• This approach ensures data integrity for critical downstream microscopy analyses.
• Enables efficient storage and analysis of terabyte-scale 4D STEM data.