Search research articles

Related Concept Videos

Super-resolution Fluorescence Microscopy

Super-resolution Fluorescence Microscopy

Super-resolution fluorescence microscopy (SRFM) provides a better resolution than conventional fluorescence microscopy by reducing the point spread function (PSF). PSF is the light intensity distribution from a point that causes it to appear blurred. Due to PSF, each fluorescing point appears bigger than its actual size, and it is the PSF interference of nearby fluorophores that causes the blurred image. Various approaches to achieving higher resolution through SRFM have recently been...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Leveraging Perspective Transformation for Enhanced Pothole Detection in Autonomous Vehicles.

Journal of imaging·2024

Same author

A Review of Detection and Removal of Raindrops in Automotive Vision Systems.

Journal of imaging·2021

Same author

Lightweight Visual Odometry for Autonomous Mobile Robots.

Sensors (Basel, Switzerland)·2018

See all related articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Video

Updated: Jul 5, 2025

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

Multispectral Deep Neural Network Fusion Method for Low-Light Object Detection.

Keval Thaker¹, Sumanth Chennupati¹, Nathir Rawashdeh²

¹Electrical and Computer Engineering, University of Michigan-Dearborn, Dearborn, MI 48128, USA.

Journal of Imaging

|January 22, 2024

Summary

Combining red, green, blue (RGB) visual and thermal infrared images with deep learning significantly improves nighttime object detection for autonomous vehicles. This multispectral approach enhances perception in low-light conditions.

Keywords:

RGB-T fusion low-light object detection multispectral fusion

More Related Videos

Deep Neural Networks for Image-Based Dietary Assessment

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

Multi-Fiber Photometry to Record Neural Activity in Freely-Moving Animals

Multi-Fiber Photometry to Record Neural Activity in Freely-Moving Animals

Published on: October 20, 2019

Related Experiment Videos

Last Updated: Jul 5, 2025

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

Deep Neural Networks for Image-Based Dietary Assessment

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

Multi-Fiber Photometry to Record Neural Activity in Freely-Moving Animals

Multi-Fiber Photometry to Record Neural Activity in Freely-Moving Animals

Published on: October 20, 2019

Area of Science:

Computer Vision
Artificial Intelligence
Robotics

Background:

Autonomous vehicle perception struggles in low-light conditions, hindering safety and reliability.
Current object detection models often rely solely on visual spectrum data, which is limited in darkness.
Multispectral imaging offers complementary data, but effective fusion for object detection remains understudied.

Purpose of the Study:

To investigate the efficacy of fusing red, green, and blue (RGB) visual and thermal infrared data for enhanced nighttime object detection.
To develop and evaluate a deep learning framework for multispectral object detection in autonomous driving scenarios.
To analyze the impact of different fusion strategies on detection performance.

Main Methods:

A deep learning framework based on the Faster R-CNN architecture with a feature pyramid network was employed.
Features from RGB visual and thermal infrared images were extracted and fused using various methods (concatenation, addition).
The proposed multispectral model was evaluated on the KAIST and FLIR datasets.

Main Results:

The multispectral object detection model significantly outperformed models using only visual (RGB) data.
Fusion of thermal and visual data provided crucial complementary information for object discrimination in low illumination.
The proposed fusion framework demonstrated superior performance compared to unimodal baseline experiments and existing multispectral detectors.

Conclusions:

Fusing visual and thermal infrared data is a highly effective strategy for improving nighttime object detection in autonomous driving.
The developed Faster R-CNN based framework with feature pyramid network and strategic fusion offers a robust solution for low-light perception challenges.
This research advances the capabilities of autonomous systems by enhancing their ability to perceive the environment under adverse lighting conditions.