Text2Makeup-DPM: A Text-Driven Dynamic Projection Mapping Makeup System for Interactive Facial Augmentation

Dynamic Projection Mapping (DPM) has revolutionized cosmetic consulting by transcending 2D screen overlays to be realized in the real world. However, existing DPM systems often rely on rigid interfaces and predefined textures, limiting creative expression. Therefore, we explore natural language as an intuitive interaction modality to maintain user focus on the projected results. Nevertheless, applying free language input remains challenging, particularly in achieving precise localized makeup generation and appropriately rendering the generated makeup on the face through projection. To address these limitations, we present Text2Makeup DPM, the first fully text-driven DPM makeup system transforming free-form descriptions into facial makeup projections. Our system features a two-scale makeup generative pipeline and efficient radiometric compensation. First, we conduct text parsing to align free-format text input into the global style and local demand of makeup. The global style module captures the overall makeup style and atmosphere through contrastive learning. A local component module optimizes latent direction compositions to enable precise, disentangled editing of specific regions (e.g., eyeshadow, lipstick). Subsequently, an asynchronous Hadamard-division compensation ensures high-fidelity projection without compromising temporal interactivity. Quantitative evaluations demonstrate that our two-scale pipeline aligns more accurately with natural language descriptions than baseline models. Furthermore, a user study (N = 12) confirms the system's effectiveness in enhancing the immersion of interaction flow, proving its contribution to intuitive DPM authoring.