Related Experiment Videos
Progressive fusion and guidance-aware denoising for robust multimodal sentiment analysis
Ziyu Zheng1, Xinfeng Ye2, Sathiamoorthy Manoharan2
1School of Computer Science, The University of Auckland, Auckland, 1010, New Zealand. zzhe232@aucklanduni.ac.nz.
Abstract:
With the rapid growth of digital media, multimodal sentiment analysis has attracted increasing research attention. However, existing approaches largely emphasize fusion strategies while overlooking the noise and redundancy inherent in real-world multimodal data, which limits the representational capacity of models. To address this issue, we propose a Progressive Fusion and Guidance-Aware Denoising Framework (PFGAD). It first learns joint multimodal representations using progressive fusion. Next, it applies a variational information bottleneck to remove noise that is irrelevant to the task. Finally, it uses a Guidance-Aware Denoising Module to make up for any information lost during denoising. This design achieves a good balance between informativeness and robustness. Our method demonstrates notable improvements on CMU-MOSI and CMU-MOSEI. On CMU-MOSI, it achieves Corr (0.814) and Acc-2 (87.5), which corresponds to a 1.75% relative gain in Corr and a 0.5%-point increase in Acc-2 over the strongest prior result. On CMU-MOSEI, it achieves Corr (0.786) and F1 (86.7), yielding a 0.77% relative gain in Corr and a 0.2%-point improvement in F1.
Related Concept Videos
Classification of Signals
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
Extraction: Advanced Methods