EmiNet: Moving bacteria detection on optical endomicroscopy images trained on synthetic data

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Respiratory Medicine

Background:

• Pneumonia diagnosis requires rapid bacterial detection, especially in critical care.
• Optical endomicroscopy (OEM) provides real-time optical biopsies for expedited bacterial identification.
• Manual analysis of OEM images is time-consuming and can delay treatment.

Purpose of the Study:

• To develop a novel deep learning model, EmiNet, for rapid segmentation and detection of bacteria in OEM images.
• To improve the efficiency and accuracy of bacterial detection in pneumonia diagnostics.
• To address the challenge of limited annotated training data for OEM bacterial images.

Main Methods:

• Proposed EmiNet, a dual-stream network combining Transformer and Convolutional Neural Networks (CNN) in an encoder-decoder architecture.
• Integrated a multi-modal cross-channel attention module to fuse appearance and motion features.
• Developed a synthetic dataset by simulating bacterial motion on real backgrounds to augment training data.
• Validated synthetic data authenticity via a Visual Turing Test with medical experts.

Main Results:

• EmiNet demonstrated superior performance compared to state-of-the-art segmentation models.
• Achieved a 6.8% improvement in detection correlation over existing bacteria detection algorithms.
• Synthetic dataset images were found to be nearly indistinguishable from real images by medical experts.

Conclusions:

• EmiNet offers a significant advancement in automated bacterial detection from OEM data.
• The developed synthetic dataset effectively compensates for the scarcity of annotated real-world data.
• EmiNet has the potential to enhance diagnostic speed and accuracy in critical care settings for pneumonia.