Dc-EEMF: Decision-level constrained multi-focus image fusion for multi-modality photoacoustic microscopy

Area of Science:

• Biomedical Optics
• Microscopy Imaging
• Artificial Intelligence in Medicine

Background:

• Multi-modality photoacoustic microscopy (mPAM) offers valuable insights into disease pathology.
• The optical-resolution mode of mPAM is limited by a shallow depth-of-field (DoF), hindering deep tissue visualization.
• Existing fusion techniques struggle to overcome DoF limitations effectively.

Purpose of the Study:

• To extend the depth-of-field (DoF) in multi-modality photoacoustic microscopy (mPAM).
• To introduce a novel decision-level constrained end-to-end multi-focus image fusion (Dc-EEMF) method.
• To enhance the diagnostic capabilities of mPAM by improving image quality and depth penetration.

Main Methods:

• Developed a decision-level constrained end-to-end multi-focus image fusion (Dc-EEMF) algorithm.
• Integrated a lightweight siamese network for feature extraction.
• Employed an artifact-resistant channel-wise spatial frequency fusion rule and a U-Net-based decision level focus perceptual loss.

Main Results:

• The Dc-EEMF method successfully extended the DoF in mPAM.
• Achieved superior image fusion performance compared to existing techniques within a single modality.
• Demonstrated strong generalizability across diverse microscopy platforms.

Conclusions:

• The proposed Dc-EEMF powered mPAM significantly overcomes the DoF limitation of traditional mPAM.
• This advanced fusion technique enhances the potential of mPAM for detailed disease pathology analysis.
• The method shows promise for broader applications in various microscopy fields.