Benchmarking deep learning for automated peak detection on GIWAXS data

Recent advancements in X-ray sources and detectors have dramatically increased data generation, leading to a greater demand for automated data processing. This is particularly relevant for real-time grazing-incidence wide-angle X-ray scattering (GIWAXS) experiments which can produce hundreds of thousands of diffraction images in a single day at a synchrotron beamline. Deep learning (DL)-based peak-detection techniques are becoming prominent in this field, but rigorous benchmarking is essential to evaluate their reliability, identify potential problems, explore avenues for improvement and build confidence among researchers for seamless integration into their workflows. However, the systematic evaluation of these techniques has been hampered by the lack of annotated GIWAXS datasets, standardized metrics and baseline models. To address these challenges, we introduce a comprehensive framework comprising an annotated experimental dataset, physics-informed metrics adapted to the GIWAXS geometry and a competitive baseline - a classical, non-DL peak-detection algorithm optimized on our dataset. Furthermore, we apply our framework to benchmark a recent DL solution trained on simulated data and discover its superior performance compared with our baseline. This analysis not only highlights the effectiveness of DL methods for identifying diffraction peaks but also provides insights for further development of these solutions.