TaskDrop: A competitive baseline for continual learning of sentiment classification.

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Continual learning presents challenges in multi-task sentiment classification due to significant cross-task similarity from shared sentiment vocabulary.
• Traditional forgetting-reduction methods are less effective because of this inherent knowledge sharing between tasks.

Purpose of the Study:

• To address the limitations of existing approaches in continual multi-task sentiment classification.
• To propose a novel method, Task-aware Dropout (TaskDrop), for efficient model capacity allocation and reuse in sequential learning tasks.

Main Methods:

• Proposed Task-aware Dropout (TaskDrop), a method that randomly samples binary masks for each task to manage model capacity.
• TaskDrop differs from standard dropout by using masks for capacity allocation and reuse across sequential tasks, rather than just for regularization.

Main Results:

• Experimental studies on Amazon review data demonstrated TaskDrop's competitive performance compared to baselines and state-of-the-art methods.
• TaskDrop showed particular effectiveness in long-term continual learning settings, highlighting the robustness of its random capacity allocation mechanism.

Conclusions:

• Task-aware Dropout (TaskDrop) is a simple yet effective mechanism for continual sentiment classification.
• The proposed random capacity allocation strategy is well-suited for handling the complexities of sequential sentiment analysis across different product categories.