A Multimodal Sensing CMOS Imager Based on Dual-Focus Imaging

Advanced machine intelligence is empowered not only by the ever-increasing computational capability for information processing but also by sensors for collecting multimodal information from complex environments. However, simply assembling different sensors can result in bulky systems and complex data processing. Herein, it is shown that a complementary metal-oxide-semiconductor (CMOS) imager can be transformed into a compact multimodal sensing platform through dual-focus imaging. By combining lens-based and lensless imaging, visual information, chemicals, temperature, and humidity can be detected with the same chip and output as a single image. As a proof of concept, the sensor is equipped on a micro-vehicle, and multimodal environmental sensing and mapping is demonstrated. A multimodal endoscope is also developed, and simultaneous imaging and chemical profiling along a porcine digestive tract is achieved. The multimodal CMOS imager is compact, versatile, and extensible and can be widely applied in microrobots, in vivo medical apparatuses, and other microdevices.