Learning to image and compute with multimode optical fibers

Multimode fibers (MMF) were initially developed to transmit digital information encoded in the time domain. There were few attempts in the late 60s and 70s to transmit analog images through MMF. With the availability of digital spatial modulators, practical image transfer through MMFs has the potential to revolutionize medical endoscopy. Because of the fiber's ability to transmit multiple spatial modes of light simultaneously, MMFs could, in principle, replace the millimeters-thick bundles of fibers currently used in endoscopes with a single fiber, only a few hundred microns thick. That, in turn, could potentially open up new, less invasive forms of endoscopy to perform high-resolution imaging of tissues out of reach of current conventional endoscopes. Taking endoscopy by its general meaning as looking into, we review in this paper novel ways of imaging and transmitting images using a machine learning approach. Additionally, we review recent work on using MMF to perform machine learning tasks. The advantages and disadvantages of using machine learning instead of conventional methods is also discussed. Methods of imaging in scattering media and particularly MMFs involves measuring the phase and amplitude of the electromagnetic wave, coming out of the MMF and using these measurements to infer the relationship between the input and the output of the MMF. Most notable techniques include analog phase conjugation [A. Yariv, "On transmission and recovery of three-dimensional image information in optical waveguides," J. Opt. Soc. Am., vol. 66, no. 4, pp. 301-306, 1976; A. Gover, C. Lee, and A. Yariv, "Direct transmission of pictorial information in multimode optical fibers," J. Opt. Soc. Am., vol. 66, no. 4, pp. 306-311, 1976; G. J. Dunning and R. Lind, "Demonstration of image transmission through fibers by optical phase conjugation," Opt. Lett., vol. 7, no. 11, pp. 558-560, 1982; A. Friesem, U. Levy, and Y. Silberberg, "Parallel transmission of images through single optical fibers," Proc. IEEE, vol. 71, no. 2, pp. 208-221, 1983], digital phase conjugation [I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, "Focusing and scanning light through a multimode optical fiber using digital phase conjugation," Opt. Express, vol. 20, no. 10, pp. 10583-10590, 2012; I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, "High-resolution, lensless endoscope based on digital scanning through a multimode optical fiber," Biomed. Opt. Express, vol. 4, no. 2, pp. 260-270, 2013] or the full-wave holographic transmission matrix method. The latter technique, which is the current gold standard, measures both the amplitude and phase of the output patterns corresponding to multiple input patterns to construct a matrix of complex numbers relaying the input to the output [Y. Choi, et al., "Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber," Phys. Rev. Lett., vol. 109, no. 20, p. 203901, 2012; A. M. Caravaca-Aguirre, E. Niv, D. B. Conkey, and R. Piestun, "Real-time resilient focusing through a bending multimode fiber," Opt. Express, vol. 21, no. 10, pp. 12881-12887; R. Y. Gu, R. N. Mahalati, and J. M. Kahn, "Design of flexible multi-mode fiber endoscope," Opt. Express, vol. 23, no. 21, pp. 26905-26918, 2015; D. Loterie, S. Farahi, I. Papadopoulos, A. Goy, D. Psaltis, and C. Moser, "Digital confocal microscopy through a multimode fiber," Opt. Express, vol. 23, no. 18, pp. 23845-23858, 2015]. This matrix is then used for imaging of the inputs or projection of desired patterns. Other techniques rely on iteratively optimizing the pixel value of the input image to perform a particular task (such as focusing or displaying an image) [R. Di Leonardo and S. Bianchi, "Hologram transmission through multi-mode optical fibers," Opt. Express, vol. 19, no. 1, pp. 247-254, 2011; T. Čižmár and K. Dholakia, "Shaping the light transmission through a multimode optical fibre: complex transformation analysis and applications in biophotonics," Opt. Express, vol. 19, no. 20, pp. 18871-18884, 2011; T. Čižmár and K. Dholakia, "Exploiting multimode waveguides for pure fibre-based imaging," Nat. Commun., vol. 3, no. 1, pp. 1-9, 2012; S. Bianchi and R. Di Leonardo, "A multi-mode fiber probe for holographic micromanipulation and microscopy," Lab Chip, vol. 12, no. 3, pp. 635-639, 2012; E. R. Andresen, G. Bouwmans, S. Monneret, and H. Rigneault, "Toward endoscopes with no distal optics: video-rate scanning microscopy through a fiber bundle," Opt. Lett., vol. 38, no. 5, pp. 609-611, 2013].