Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop Alignment

Area of Science:

• Computer Vision
• Computer Graphics
• Robotics

Background:

• Markerless projection mapping is challenging due to calibration requirements.
• Real-time dynamic content projection onto arbitrary surfaces demands high frame rates and robust tracking.

Purpose of the Study:

• To develop a fast, markerless projection mapping method for dynamic video content.
• To eliminate the need for camera-projector calibration and manual adjustments.
• To enable seamless video projection onto textured, non-planar surfaces in real-time.

Main Methods:

• A closed-loop alignment approach using a camera to track surface texture and projected images simultaneously.
• Integration of fiducial patterns within a fast-flapping binary frame sequence from a Digital Micromirror Device (DMD) projector.
• Simultaneous tracking of surface texture and fiducial geometry from a single camera image.

Main Results:

• The method achieves a processing speed of 400 frames per second (fps) on a CPU.
• Successfully demonstrates "sticking" arbitrary video content onto various textured surfaces without prior calibration.
• The closed-loop system autonomously handles alignment, removing the need for manual adjustments.

Conclusions:

• The proposed fast projection mapping method offers a robust and efficient solution for dynamic content.
• Eliminates calibration complexities, making projection mapping more accessible for real-world applications.
• Enables novel applications in augmented reality and interactive displays by seamlessly integrating digital content with physical environments.