TCA-EfficientSCI: A Lightweight Causal Baseline for Cross-Measurement Temporal Continuity in Snapshot Compressive...

Area of Science:

• Computer Vision
• Signal Processing
• Image Reconstruction

Background:

• Snapshot compressive imaging (SCI) reconstructs high-speed videos from compressed measurements.
• Current deep learning models often reconstruct frames independently, neglecting temporal continuity.
• This leads to artifacts like flickering and motion jumps between segments.

Purpose of the Study:

• To address the underexplored problem of cross-measurement temporal continuity in continuous SCI.
• To introduce TCA-EfficientSCI, a lightweight, causal baseline for improved temporal coherence.
• To analyze the impact of temporal context on SCI reconstruction quality.

Main Methods:

• Developed a Temporal Context Adapter (TCA) using past reconstructed frames as causal context.
• Integrated TCA via a gated residual pathway into an EfficientSCI network.
• Implemented a boundary consistency loss to regularize temporal variations across measurement boundaries.

Main Results:

• Full TCA with boundary loss reduced mean Boundary Difference Error (BDE) by 2.23% compared to EfficientSCI.
• Correct temporal history yielded lower BDE (0.01615) than zero or shuffled history.
• The adapter increased model parameters by 11.56% and latency by ~14ms per frame.

Conclusions:

• Cross-measurement temporal continuity is a critical dimension for continuous SCI.
• TCA-EfficientSCI provides a viable baseline for enhancing temporal coherence in SCI.
• Future SCI designs should prioritize modeling temporal dependencies for artifact reduction.