Random access vision: an imaging method to observe arbitrary and multiple gaze directions in frame-by-frame manner

In robot vision, it is often desired to measure an area larger than the field of view of the camera, so the camera tends to be mounted on a mechanical pan/tilt platform. However, such mechanisms have a non-negligible response time compared to the frame rate of the camera. In this paper, we describe what we believe to be a new method that allows arbitrary and multiple gaze directions to be observed in a frame-by-frame manner based on a resonant mirror and a lock-in pixel image sensor. In the proposed method, while the gaze direction oscillates due to the resonance mirror, the image sensor makes an exposure of several hundreds of nanoseconds every time the gaze passes through the direction to be captured, and accumulates the captured signals. A prototype system was developed using a lock-in pixel image sensor with four image storage units called taps and a resonant mirror with a resonant frequency of 12 kHz. The system achieved both arbitrary control of the gaze direction in a frame-by-frame manner, and simultaneous capturing of four images with arbitrary gaze directions also in a frame-by-frame manner.