Stereoscopic artificial compound eyes for spatiotemporal perception in three-dimensional space

Area of Science:

• Biomimetic engineering
• Robotics and computer vision
• Neuroscience and sensory systems

Background:

• Arthropod eyes excel at object tracking and wide field of view but lack depth perception for static objects.
• Most arthropods use motion parallax for 3D object tracking, unlike mammals.
• Praying mantises uniquely combine compound eyes and stereopsis for 3D object recognition.

Purpose of the Study:

• To mimic the praying mantis vision system using artificial compound eyes for 3D spatiotemporal object sensing and tracking.
• To develop an energy-efficient system with minimal latency for visual information processing.
• To explore hardware and software co-design for edge computing and sensing capabilities.

Main Methods:

• Designed and fabricated stereoscopically coupled artificial compound eyes inspired by praying mantis vision.
• Implemented edge processing using a synaptic device and a federated split learning algorithm.
• Evaluated spatiotemporal object sensing and optical flow tracking accuracy and energy consumption.

Main Results:

• The biomimetic imager achieved accurate spatiotemporal object sensing and tracking in 3D space with a wide field of view.
• The system demonstrated a root mean square error of 0.3 cm for tracking.
• Energy consumption was approximately 4 millijoules, over 400 times lower than conventional CMOS imaging systems.

Conclusions:

• The stereoscopic artificial compound eye effectively replicates praying mantis vision for 3D object sensing and tracking.
• Edge processing with synaptic devices and federated learning enables energy-efficient, low-latency visual sensing.
• This biomimetic approach highlights the potential of integrating natural designs with co-designed technology for advanced edge computing and sensing.