Homomorphic data compression for real time photon correlation analysis

Area of Science:

• Coherent X-ray science
• Materials science
• Data science

Background:

• Advancements in coherent X-ray sources and detectors enable new research but create data challenges.
• X-ray photon correlation spectroscopy (XPCS) particularly benefits from faster acquisition, enabling study of faster dynamics.
• Real-time analysis for XPCS requires simultaneous computation on extensive time-series data, posing significant processing and storage hurdles.

Purpose of the Study:

• To introduce a novel homomorphic compression scheme for X-ray photon correlation spectroscopy (XPCS) data.
• To reduce computational time and memory requirements for XPCS analysis.
• To enable real-time data processing and efficient data storage for advanced X-ray experiments.

Main Methods:

• Developed a homomorphic compression scheme leveraging mathematical similarities between matrix compression and correlation calculations.
• Implemented a method allowing direct computation on compressed data without decompression.
• Designed both offline compression for storage and online compression for real-time processing.

Main Results:

• Offline compression achieved a 40x increase in storage capacity while preserving essential data features.
• Online compression reduced computational time to below 1 millisecond, enabling kHz framerate correlation function calculations.
• Demonstrated effective reduction of computational and storage burdens for XPCS analysis.

Conclusions:

• The homomorphic compression scheme offers an effective solution to the big data challenges at coherent light sources.
• This framework facilitates real-time operations directly on compressed data streams, applicable beyond XPCS.
• The method enhances the feasibility of studying faster dynamics in materials using advanced X-ray techniques.