Multi-source sequential knowledge regression by using transfer RNN units.

Area of Science:

• Machine Learning
• Deep Learning
• Artificial Intelligence

Background:

• Transfer learning in deep neural networks is successful for classification but limited in multi-source regression.
• Existing methods struggle with common latent feature learning and source information loss in regression tasks.

Purpose of the Study:

• To propose novel transferable Recurrent Neural Network (RNN) units for Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models.
• To adapt source knowledge effectively in multi-source regression scenarios.

Main Methods:

• Introduced transferable RNN units integrated into LSTM and GRU architectures.
• Proposed two knowledge adaptation methods: similarity weights and a transfer-gate.
• Adapted source knowledge from both internal states and outputs of the recurrent units.

Main Results:

• Demonstrated superior performance of the proposed transfer RNN units over conventional models.
• Achieved effective adaptation of source knowledge in multi-source regression.
• Validated through extensive experiments on synthetic data and human motion prediction (Human 3.6M dataset).

Conclusions:

• The proposed transferable RNN units significantly improve multi-source knowledge regression.
• The novel adaptation methods effectively address limitations of existing transfer learning strategies in regression.
• This work offers a promising direction for knowledge transfer in complex regression tasks.