Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Super-resolution Fluorescence Microscopy01:37

Super-resolution Fluorescence Microscopy

7.0K
Super-resolution fluorescence microscopy (SRFM) provides a better resolution than conventional fluorescence microscopy by reducing the point spread function (PSF). PSF is the light intensity distribution from a point that causes it to appear blurred. Due to PSF, each fluorescing point appears bigger than its actual size, and it is the PSF interference of nearby fluorophores that causes the blurred image. Various approaches to achieving higher resolution through SRFM have recently been...
7.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Leveraging Perspective Transformation for Enhanced Pothole Detection in Autonomous Vehicles.

Journal of imaging·2024
Same author

A Review of Detection and Removal of Raindrops in Automotive Vision Systems.

Journal of imaging·2021
Same author

Lightweight Visual Odometry for Autonomous Mobile Robots.

Sensors (Basel, Switzerland)·2018
See all related articles

Related Experiment Video

Updated: Jul 5, 2025

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
03:31

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

556

Multispectral Deep Neural Network Fusion Method for Low-Light Object Detection.

Keval Thaker1, Sumanth Chennupati1, Nathir Rawashdeh2

  • 1Electrical and Computer Engineering, University of Michigan-Dearborn, Dearborn, MI 48128, USA.

Journal of Imaging
|January 22, 2024
PubMed
Summary

Combining red, green, blue (RGB) visual and thermal infrared images with deep learning significantly improves nighttime object detection for autonomous vehicles. This multispectral approach enhances perception in low-light conditions.

Keywords:
RGB-T fusionlow-light object detectionmultispectral fusion

More Related Videos

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.2K
Multi-Fiber Photometry to Record Neural Activity in Freely-Moving Animals
05:52

Multi-Fiber Photometry to Record Neural Activity in Freely-Moving Animals

Published on: October 20, 2019

36.2K

Related Experiment Videos

Last Updated: Jul 5, 2025

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
03:31

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

556
Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.2K
Multi-Fiber Photometry to Record Neural Activity in Freely-Moving Animals
05:52

Multi-Fiber Photometry to Record Neural Activity in Freely-Moving Animals

Published on: October 20, 2019

36.2K

Area of Science:

  • Computer Vision
  • Artificial Intelligence
  • Robotics

Background:

  • Autonomous vehicle perception struggles in low-light conditions, hindering safety and reliability.
  • Current object detection models often rely solely on visual spectrum data, which is limited in darkness.
  • Multispectral imaging offers complementary data, but effective fusion for object detection remains understudied.

Purpose of the Study:

  • To investigate the efficacy of fusing red, green, and blue (RGB) visual and thermal infrared data for enhanced nighttime object detection.
  • To develop and evaluate a deep learning framework for multispectral object detection in autonomous driving scenarios.
  • To analyze the impact of different fusion strategies on detection performance.

Main Methods:

  • A deep learning framework based on the Faster R-CNN architecture with a feature pyramid network was employed.
  • Features from RGB visual and thermal infrared images were extracted and fused using various methods (concatenation, addition).
  • The proposed multispectral model was evaluated on the KAIST and FLIR datasets.

Main Results:

  • The multispectral object detection model significantly outperformed models using only visual (RGB) data.
  • Fusion of thermal and visual data provided crucial complementary information for object discrimination in low illumination.
  • The proposed fusion framework demonstrated superior performance compared to unimodal baseline experiments and existing multispectral detectors.

Conclusions:

  • Fusing visual and thermal infrared data is a highly effective strategy for improving nighttime object detection in autonomous driving.
  • The developed Faster R-CNN based framework with feature pyramid network and strategic fusion offers a robust solution for low-light perception challenges.
  • This research advances the capabilities of autonomous systems by enhancing their ability to perceive the environment under adverse lighting conditions.