Distributed X-ray photon correlation spectroscopy data reduction using Hadoop MapReduce
Faisal Khan1, Suresh Narayanan1, Roger Sersted2
1X-ray Science Division, Argonne National Laboratory, 9700 South Cass Avenue, Lemont, IL 60439, USA.
Abstract:
Multi-speckle X-ray photon correlation spectroscopy (XPCS) is a powerful technique for characterizing the dynamic nature of complex materials over a range of time scales. XPCS has been successfully applied to study a wide range of systems. Recent developments in higher-frame-rate detectors, while aiding in the study of faster dynamical processes, creates large amounts of data that require parallel computational techniques to process in near real-time. Here, an implementation of the multi-tau and two-time autocorrelation algorithms using the Hadoop MapReduce framework for distributed computing is presented. The system scales well with regard to the increase in the data size, and has been serving the users of beamline 8-ID-I at the Advanced Photon Source for near real-time autocorrelations for the past five years.
Related Concept Videos
Correlation of Experimental Data
For example, a spherical particle moving through a viscous fluid experiences drag. Dimensional analysis shows that the drag force depends on the particle's diameter, velocity,...
Data: Types and Distribution
Distributions in...
Correlations
2D NMR: Homonuclear Correlation Spectroscopy (COSY)
X-ray Crystallography
Diffraction
Diffraction is the change in the direction of travel experienced by an electromagnetic wave when it encounters a physical barrier whose dimensions are comparable to those of the wavelength of the light. X-rays are electromagnetic radiation with wavelengths about as long as the distance between neighboring...
Correlation and Causation
Correlation versus Causation
If the dependent variable increases or decreases when the independent variable increases, there is a positive or negative...


