Related Experiment Video
Updated: Jul 16, 2026
04:43
Visualizing Visual Adaptation
Published on: April 24, 2017
Enhancing Perception Through Context-Adaptive Visible and SWIR Image Fusion in Harsh Environments
Alexandre Riffard1, Mathieu Labussière1, Pierre Duthon2
1Université Clermont Auvergne, Clermont Auvergne INP, CNRS, Institut Pascal, F-63000 Clermont-Ferrand, France.
Sensors (Basel, Switzerland)
|July 15, 2026
Summary
Autonomous vehicles struggle in bad weather. This study introduces VISWIR, a new fusion method using visible and short-wave infrared sensors to improve perception in fog, rain, and snow.
Area of Science:
- Computer Vision
- Sensor Fusion
- Autonomous Systems
Background:
- Autonomous vehicle perception is challenged by adverse weather conditions like fog, rain, and snow.
- Short-wave infrared (SWIR) sensors can penetrate atmospheric disturbances, but fusing their data with visible (VIS) cameras is complex due to signal decorrelation and static fusion limitations.
Purpose of the Study:
- To develop a robust and lightweight pixel-level image fusion method for enhancing autonomous vehicle perception in adverse weather.
- To address the limitations of static fusion schemes by introducing an adaptive parameter scheduling strategy.
Main Methods:
- Proposed VISWIR (Visible and SWIR Weighted Image Reconstruction), a pixel-level fusion method utilizing a multi-scale pyramid architecture.
- Implemented an automated parameter scheduling strategy based on weather conditions within an optimization framework.
- Employed a multi-objective optimization approach maximizing perceptual image quality using No-Reference Image Quality Assessment (NR-IQA) metrics.
Main Results:
- VISWIR demonstrated effective fusion of VIS and SWIR data, enhancing image quality in simulated adverse weather.
- The automated parameter scheduling adapted fusion hyperparameters based on meteorological context, outperforming static fusion.
- Validated in controlled scenarios with varying weather severities, confirming the method's robustness.
Conclusions:
- VISWIR offers a promising algorithmic baseline for improving autonomous vehicle perception in challenging weather conditions.
- The adaptive fusion approach enhances robustness and perceptual image quality, crucial for safe autonomous driving.
- This lightweight method has the potential to significantly advance the reliability of autonomous systems in real-world environments.
Related Concept Videos
02:16Light Acquisition
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
01:15Depth Perception and Spatial Vision
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
01:12Perceptual Constancy
Perceptual constancy is the ability to recognize that objects remain consistent and unchanged even when their appearance varies due to changes in sensory input. There are four main types of perceptual constancy: size constancy, shape constancy, color constancy, and brightness constancy.
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
01:24Vision
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
01:26Visual System
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
01:28Perception
Perception is a fundamental psychological process that enables individuals to organize, interpret, and consciously experience sensory information. This process is crucial for understanding and interacting with the world around us. It includes both bottom-up and top-down processing, each playing a distinct role in how we perceive our environment.
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...