Flexible artificial compound eye cameras for ultrawide continuous tracking in mixed reality

In nature, fruit flies Drosophila have evolved a simple but superb vision system characterised by a three-level synergy: natural compound eyes for panoramic perception, head muscles for continuous tracking in dim conditions, and neural circuits for dynamic scenes. This vision system serves as an ideal model for biomimicry, yet achieving true replication remains challenging, despite significant progress in artificial compound eyes recently. In this study, we present a flexible artificial compound eye camera that adopts such a three-level synergy. An artificial compound eye is constructed by plastic optical fibres and curved microlens arrays for real-time panoramic imaging. Two tethers simulate head muscle movements, achieving a 270° field of view for continuous tracking of weak signals. An artificial intelligence algorithm mimics neural processing, enabling the reconstruction of interactive mixed-reality scenes at rates up to 7000 fps. This unique integration of panoramic sensing, active tracking, and neural-like processing establishes a framework for vision-based metaverse applications and bioinspired wearable technologies.