Promise and pitfalls of random projection algorithms for optical processors

Optical computing is an emerging field with the potential to enhance the performance and energy efficiency of artificial intelligence-related computing. One prospective application of optics in computation is random projection. Random projection, characterized by the use of a Gaussian independently and identically distributed (i.i.d.) matrix, is notable for its simplicity and ability to preserve distances within high-dimensional space. This property is leveraged by several machine learning algorithms to improve computational efficiency, including randomized singular value decomposition (RSVD), no-prior-knowledge exponentially weighted moving average (NEWMA) for change-point detection, and reservoir computing for time-series forecasting. Moreover, the application of random projection extends to neural network training methodologies such as direct feedback alignment (DFA). Optical implementations of random projection have also been explored, offering potential benefits in terms of time efficiency and energy savings, with initiatives like LightOn leading such efforts. This article provides a comprehensive analysis of these algorithms and identifies a trend in which random projection accounts for a small fraction of the total computational time in most algorithms examined. Additionally, we highlight the limitations of optical random projection, demonstrating that it does not significantly enhance performance. This finding raises questions about the relevance of random projection-based optical processing units.