A Modified Long Short-Term Memory Cell.

Area of Science:

• Artificial Intelligence
• Natural Language Processing

Background:

• Machine Learning (ML) enhances text classification through models like Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM), Gated Recurrent Units (GRUs), and Transformer Models.
• LSTM cells possess internal memory states ('Current' and 'Hidden') crucial for dynamic temporal behavior in sequence data.

Purpose of the Study:

• To introduce and evaluate a novel modification layer within the LSTM cell architecture.
• To explore the impact of altering 'Current' and 'Hidden' states on text classification performance.

Main Methods:

• Implemented 17 distinct state alteration experiments within the LSTM cell, focusing on 'Current' and 'Hidden' states.
• Evaluated the modified LSTM cell across seven diverse datasets, including sentiment analysis, document classification, hate speech detection, and human-to-robot interaction.
• Compared the performance of the modified LSTM against two Transformer models.

Main Results:

• The highest performing alterations yielded average F1 score improvements of 0.5% for the 'Current' state and 0.3% for the 'Hidden' state.
• The modified LSTM cell was outperformed by Transformer models in 4 out of 6 classification datasets.
• The modified LSTM demonstrated superior cost efficiency compared to both evaluated Transformer models.

Conclusions:

• Modifying LSTM cell states can lead to marginal performance gains in specific text classification tasks.
• While not surpassing advanced Transformer models in all metrics, the modified LSTM offers a compelling cost-efficiency advantage.