A novel activation function based recurrent neural networks and their applications on sentiment classification and...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Recurrent Neural Networks (RNNs) are crucial for sequential data processing.
• Existing activation functions may limit RNN performance in complex tasks.
• Efficient dynamic system modeling remains a challenge.

Purpose of the Study:

• To introduce a novel nonlinear activation function (NAF).
• To enhance RNN models for sentiment classification and dynamic systems.
• To develop and validate a fixed-time convergent RNN (FTCRNN) model.

Main Methods:

• Proposed a new nonlinear activation function (NAF).
• Constructed Simple RNN (SRNN), LSTM, and GRU models with NAF for sentiment classification.
• Developed an FTCRNN model incorporating the NAF.
• Validated FTCRNN convergence properties and derived convergence time formulas.
• Applied FTCRNN to Dynamic Sylvester Equation (DSE) solving, robot trajectory tracking, and circuit current computation.

Main Results:

• NAF-enhanced RNN models (SRNN, LSTM, GRU) outperformed standard activation functions on IMDB sentiment classification.
• FTCRNN model demonstrated rapid and accurate convergence for DSE solving, even with noise.
• FTCRNN showed superior performance and robustness in robot manipulator trajectory tracking and electric circuit current computation.

Conclusions:

• The proposed NAF significantly improves RNN performance for sentiment analysis.
• The FTCRNN model offers fast, reliable, and robust solutions for dynamic systems.
• NAF and FTCRNN exhibit broad applicability and superior performance across diverse computational tasks.