Dynamic Projection Mapping via Fast 3D Gaussian Splatting with Blue-Noise Stochastic Sampling

Dynamic projection mapping (DPM) enables spatially aligned projection onto moving objects, but requires low-latency rendering and accurate appearance reproduction. 3D Gaussian splatting (3DGS) has recently emerged as an image-based approach for reconstructing 3D scenes solely from photographs and rendering them from arbitrary viewpoints. This paper proposes a framework for realizing DPM using 3DGS. Whereas conventional DPM approaches based on physically based rendering require material parameters to be obtained in advance, the proposed 3DGS-based framework reproduces real object appearance with high fidelity directly from multi-view images, offering a simpler data acquisition workflow. However, the low-latency rendering required for DPM necessitates the use of fast 3DGS variants, which introduce rendering noise that leads to visible projection artifacts. We introduce blue-noise sampling into stochastic-transparency-based 3DGS to concentrate the noise in high spatiotemporal frequencies, where the human visual system is less sensitive and temporal accumulation is effective. We further reduce the noise by combining temporal anti-aliasing, high-frame-rate projection that leverages persistence of vision, and checkerboard rendering. Finally, assuming a rigid target, we exploit the equivalence between target motion in DPM and viewpoint motion in 3DGS to obtain viewpoints consistent with the pose of the projection target, enabling the direct use of existing 3DGS renderers. In experiments on synthetic scenes and real-world projections, the proposed method reduces LPIPS by 57.7% and 45.2%, respectively, compared with a conventional fast variant, while maintaining comparable rendering speed.