Lifelong Text-Audio Sentiment Analysis learning.

Area of Science:

• Natural Language Processing
• Multimodal Machine Learning
• Artificial Intelligence

Background:

• Existing sentiment analysis methods primarily focus on text, often overlooking valuable information in other modalities like audio.
• Current approaches struggle with continuous learning of new sentiment analysis tasks and discovering cross-modal correlations.

Purpose of the Study:

• To propose a novel Lifelong Text-Audio Sentiment Analysis (LTASA) model for continuous learning of sentiment analysis tasks.
• To effectively explore semantic relationships within and between text and audio modalities.
• To address limitations in existing sentiment analysis models regarding multimodal integration and lifelong learning.

Main Methods:

• Developed modality-specific knowledge dictionaries for shared intra-modality representations across tasks.
• Created a complementarity-aware subspace to capture nonlinear inter-modality knowledge based on information dependence.
• Designed an online multi-task optimization pipeline for sequential learning of text-audio sentiment analysis tasks.

Main Results:

• The LTASA model demonstrated superior performance on three common datasets compared to baseline methods.
• Significantly improved sentiment analysis capabilities were observed across five key measurement indicators.
• The model effectively leveraged both intra-modality and inter-modality information for enhanced sentiment detection.

Conclusions:

• The proposed LTASA model offers a robust framework for lifelong multimodal sentiment analysis.
• Integrating text and audio data through specialized dictionaries and subspaces enhances sentiment analysis accuracy.
• The model's ability to continuously learn and discover cross-modal correlations represents a significant advancement in the field.