Tri-attention complex-valued CNN for multimodal holographic digital human interaction

Holographic displays show great potential in immersive applications such as AR/VR; however, it still faces challenges in the generation quality of full-color computer-generated holography (CGH), visual perception consistency, and real-time interaction. This paper proposes a multimodal voice-interactive holographic digital human avatar system. The system relies on an RGB-D digital human avatar generated by AIGC technology, enabling personalized configurations such as outfit changes, personality simulation, and voice cloning, thus realizing end-to-end RGB-D generation. We adopt an end-to-end semantic interaction architecture of "ASR + Dual-LLM + TTS" to construct a multi-agent interactive system, achieving high-quality real-time voice interaction. To achieve high-fidelity full-color holographic reconstruction, we propose a Tri-Attention Complex-valued Convolutional Neural Network (TA-CCNN). This network introduces a third-dimensional Energy-Aware branch to build a three-branch parallel attention mechanism, which significantly enhances the contrast in high-energy regions (such as the eyes and facial features), improving color reproduction and ultimately presenting superior visual effects with deep backgrounds and vivid colors.