Color Night Vision: Opponent Processing in the Fusion of Visible and IR Imagery

We describe here a means of fusing registered low-light visible and thermal infrared (IR) imagery to support realtime color night vision. Opponent processing, in the form of feedforward center-surround shunting neural networks, is used to contrast enhance and adaptively normalize both visible and IR imagery separately. Both positive and negative polarity ("on" and "off") enhanced IR imagery is then combined with the enhanced visible imagery to create two single-opponent color-contrast grayscale images. The opponent processed visible and opponent-color images (forming a set of three grayscale images) are then assigned directly to the red, green, blue (RGB) color space. Final manipulation of both hue and saturation is achieved in the hue, saturation, value (HSV) color space. Remarkably realistic color renderings of night scenes are obtained which may support perceptual "pop-out" of extended navigation cues and compact targets. Psychophysical testing on low contrast targets in natural dynamic scenes is called for in order to assess human performance using fused visible and IR imagery at night. Copyright 1996 Elsevier Science Ltd.