The reproducibility gap in graph neural network workflows for cell dynamics: A checklist-driven case study

Area of Science:

• Quantitative Biology
• Bioimage Analysis
• Computational Microscopy

Background:

• The Global BioImage Analysts' Society (GloBIAS) initiative aims to improve reproducibility in biological image analysis.
• The Quality Assessment and Reproducibility for Instruments and Images in Light Microscopy (QUAREP-LiMi) initiative provides checklists for assessing reproducibility.
• This study retrospectively evaluates a 2022 Graph Neural Network (GNN) paper using QUAREP-LiMi standards.

Purpose of the Study:

• To assess the reproducibility of a GNN study on cell dynamics.
• To identify gaps between current reporting standards and practical execution of computational workflows.
• To extract lessons for establishing future research standards in quantitative biology.

Main Methods:

• Utilized structured, community-developed checklists from the QUAREP-LiMi initiative.
• Performed reproduction attempts of the GNN study across multiple computational environments.
• Retrospectively analyzed a legacy computational paper against modern reproducibility frameworks.

Main Results:

• Identified significant reproducibility deficiencies in the target GNN study.
• Confirmed the absence of crucial image metadata (pixel size, timestamps), limiting quantitative interpretation.
• Observed challenges in environment setup due to missing software containers and incomplete dependency lists, increasing complexity.

Conclusions:

• A gap exists between contemporary reproducibility expectations and the practical implementation in published research.
• Robust environment containerization and standardized data deposition are crucial for ensuring the scientific soundness and reusability of complex computational workflows.
• This retrospective analysis provides actionable recommendations for advancing reproducibility standards in quantitative biology.