Geometrically-correct projection-based texture mapping onto a deformable object.

Area of Science:

• Computer Vision
• Augmented Reality
• Human-Computer Interaction

Background:

• Traditional projection-based augmented reality (AR) systems are limited to rigid projection surfaces.
• Existing systems cannot support applications involving deformable substrates that can be reshaped (bent, twisted, folded).
• There is a need for AR systems that can accurately map projections onto non-rigid, dynamic surfaces.

Purpose of the Study:

• To develop a projection-based AR system that supports deformable substrates.
• To enable geometrically correct texture projection onto objects that undergo significant shape changes.
• To address the challenge of robustly recognizing and tracking markers on deforming surfaces.

Main Methods:

• Development of a novel invisible marker embedded within a deformable substrate.
• Implementation of an algorithm for identifying substrate deformations using 3D shape measurement via retro-reflective marker detection.
• Proposal of a partially recognizable marker pattern for accurate registration and texture mapping onto the deforming object's surface.

Main Results:

• Demonstration of a projection-based AR system that successfully supports bent, twisted, or folded substrates.
• Achieved geometrically correct projection-based texture mapping onto deformable objects under extreme deformation conditions.
• Validated the system's ability to recognize and track the underlying material despite significant shape changes.

Conclusions:

• The presented techniques overcome fundamental vision recognition challenges for AR on deformable materials.
• The system enables accurate texture projection onto dynamic, non-rigid surfaces, expanding AR capabilities.
• Potential applications include prototype development, design, entertainment, and information-based AR systems.