Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Endoscopic Procedures III: Video Capsule Endoscopy01:28

Endoscopic Procedures III: Video Capsule Endoscopy

335
Capsule endoscopy, or wireless or video capsule endoscopy, is a diagnostic procedure for examining the entire gastrointestinal tract. Patients swallow a capsule about the size of a vitamin tablet. The capsule is equipped with a transmitter, a battery, an LED light source, and a color video camera to capture images throughout the gastrointestinal tract. This procedure is particularly useful for diagnosing conditions such as Crohn's disease, ulcerative colitis, tumors, polyps, ulcers,...
335
  1. Home
  3. Research Domains
  5. Information And Computing Sciences
  7. Computer Vision And Multimedia Computation
  9. Video Processing
  11. Encapsulate: Neural Cellular Automata For Precision Diagnosis On Capsule Endoscopes

eNCApsulate: neural cellular automata for precision diagnosis on capsule endoscopes

Henry John Krumb1, Anirban Mukhopadhyay2

  • 1Computer Science Department, TU Darmstadt, Fraunhoferstr. 5, Darmstadt, 64293, Hessen, Germany. henry_john.krumb@tu-darmstadt.de.

International Journal of Computer Assisted Radiology and Surgery
|July 4, 2025

Related Experiment Videos

Automated Multimodal Stimulation and Simultaneous Neuronal Recording from Multiple Small Organisms
08:28

Automated Multimodal Stimulation and Simultaneous Neuronal Recording from Multiple Small Organisms

Published on: March 3, 2023

1.2K
Application of Automated Image-guided Patch Clamp for the Study of Neurons in Brain Slices
09:05

Application of Automated Image-guided Patch Clamp for the Study of Neurons in Brain Slices

Published on: July 31, 2017

11.7K
Photodiode-Based Optical Imaging for Recording Network Dynamics with Single-Neuron Resolution in Non-Transgenic Invertebrates
10:18

Photodiode-Based Optical Imaging for Recording Network Dynamics with Single-Neuron Resolution in Non-Transgenic Invertebrates

Published on: July 9, 2020

3.0K

View abstract on PubMed

Summary
This summary is machine-generated.

Neural cellular automata (NCAs) enable on-capsule bleeding segmentation and depth estimation for wireless capsule endoscopy (WCE). This miniaturized AI approach enhances diagnostic precision and capsule localization, overcoming limitations of large deep learning models.

Area of Science:

  • Medical Imaging
  • Artificial Intelligence
  • Miniaturized Electronics

Background:

  • Wireless capsule endoscopy (WCE) offers a noninvasive method for gastrointestinal imaging, but generates vast data, challenging capsule localization and pathology identification.
  • Existing deep learning models for tasks like bleeding detection and depth estimation are often too large for direct implementation on capsule hardware.
  • Accurate localization of pathologies and the capsule itself is crucial for effective diagnosis and treatment planning in WCE.

Purpose of the Study:

  • To develop and implement lightweight deep learning models, specifically Neural Cellular Automata (NCAs), for on-capsule bleeding segmentation and depth estimation.
  • To enable efficient image processing directly on miniaturized hardware within a wireless capsule endoscope.
  • To improve the precision of diagnosis and localization capabilities of WCE systems.
Keywords:
Depth estimationEdge AIImage segmentationNeural cellular automata

Related Experiment Videos

Automated Multimodal Stimulation and Simultaneous Neuronal Recording from Multiple Small Organisms
08:28

Automated Multimodal Stimulation and Simultaneous Neuronal Recording from Multiple Small Organisms

Published on: March 3, 2023

1.2K
Application of Automated Image-guided Patch Clamp for the Study of Neurons in Brain Slices
09:05

Application of Automated Image-guided Patch Clamp for the Study of Neurons in Brain Slices

Published on: July 31, 2017

11.7K
Photodiode-Based Optical Imaging for Recording Network Dynamics with Single-Neuron Resolution in Non-Transgenic Invertebrates
10:18

Photodiode-Based Optical Imaging for Recording Network Dynamics with Single-Neuron Resolution in Non-Transgenic Invertebrates

Published on: July 9, 2020

3.0K

Main Methods:

  • Trained NCAs on capsule endoscopic images for bleeding segmentation and monocular depth estimation.
  • Utilized knowledge distillation, using a large foundation model's outputs as pseudo-ground truth, to train a compact NCA architecture.
  • Ported trained NCA models to the ESP32 microcontroller for efficient on-device image processing.

Main Results:

  • NCAs demonstrated superior accuracy (Dice score) for segmentation compared to other portable models, with over 100x fewer memory parameters.
  • NCA-based depth estimation produced visually convincing results, matching or exceeding the realism and detail of the pseudo-ground truth.
  • Runtime optimizations on the ESP32-S3 microcontroller led to a significant acceleration of average inference speed (over 3x).

Conclusions:

  • Algorithmic adjustments and distillation successfully enabled the encapsulation of NCA models into microcontrollers suitable for WCE.
  • This study presents the first successful implementation of reliable bleeding segmentation and depth estimation directly on a miniaturized WCE device.
  • The developed technology paves the way for precise diagnosis and visual odometry-based capsule localization within the gastrointestinal tract.
Wireless capsule endoscopy