RocSync: Millisecond-Accurate Temporal Synchronization for Heterogeneous Camera Systems

Area of Science:

• Computer Vision
• Sensor Fusion
• Signal Processing

Background:

• Accurate spatiotemporal alignment of multi-view video is crucial for dynamic scene analysis, including 3D reconstruction and pose estimation.
• Synchronizing heterogeneous camera systems (e.g., professional/consumer, RGB/IR) is challenging due to unavailable hardware synchronization, especially in real-world scenarios.

Purpose of the Study:

• To develop a low-cost, general-purpose method for achieving millisecond-level temporal alignment across diverse camera systems.
• To support synchronization for both visible (RGB) and infrared (IR) modalities in unconstrained environments.

Main Methods:

• A custom-built LED Clock was designed to encode time using red and infrared LEDs.
• The LED Clock's time encoding allows visual decoding of exposure window start and end times from recorded frames.
• The method was benchmarked against hardware synchronization and other synchronization techniques (light, audio, timecode).

Main Results:

• Achieved millisecond-level temporal alignment with a residual error of 1.34 ms RMSE across multiple recordings when compared to hardware synchronization.
• Outperformed light-, audio-, and timecode-based synchronization methods in experimental evaluations.
• Demonstrated direct improvement in downstream computer vision tasks, such as multi-view pose estimation and 3D reconstruction.

Conclusions:

• The proposed LED Clock system offers a simplified and streamlined solution for camera synchronization in unconstrained environments.
• This method enhances accessibility to advanced vision-based sensing for industrial and clinical applications, including large-scale surgical recordings.
• The system effectively synchronizes over 25 heterogeneous cameras across IR and RGB modalities.