Automatic consecutive context perceived transformer GAN for serial sectioning image blind inpainting.

Area of Science:

• Histology and Digital Image Processing
• Computational Pathology
• Medical Imaging Analysis

Background:

• Serial sectioning is a standard histology technique, often yielding defective images.
• Manual annotation of broken areas in serial section image stacks is labor-intensive and limits research.
• There is a need for automated methods to restore and analyze these image stacks.

Purpose of the Study:

• To develop a fully automatic approach for locating and restoring defective serial section image stacks.
• To overcome the limitations of manual annotation in histological image analysis.
• To improve the efficiency and accuracy of analyzing large-scale histological image data.

Main Methods:

• Proposed a novel end-to-end framework: automatic consecutive context perceived transformer GAN (ACCP-GAN).
• Employed a two-stage network: auto-detection module for rough repair and a refined inpainting module for precise patch generation.
• Integrated a self-attention-based SPTransformer for enhanced feature extraction from neighboring images and utilized gated convolution.

Main Results:

• Achieved high accuracy in broken area segmentation (0.9995).
• Demonstrated superior restoration performance with metrics FSIM=0.9478, MS-SSIM=0.9592, PSNR=29.7903, VIF=0.8543, and FID=47.2252.
• Showcased satisfying generalization ability on independent datasets (E17, N5).

Conclusions:

• The ACCP-GAN method successfully detects and restores defective serial sectioning image stacks automatically, including large broken patches.
• The SPTransformer module effectively extracts relevant features, improving restoration quality.
• This automated approach significantly reduces manual workload and enhances the analysis of histological image stacks.