autoBOT: evolving neuro-symbolic representations for explainable low resource text classification.

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• State-of-the-art neural language models excel at text classification but are resource-intensive.
• Training large models requires substantial data, computational power, and extensive hyperparameter tuning.
• Existing methods pose challenges for low-resource learning environments with limited data and hardware.

Purpose of the Study:

• To develop an explainable, low-resource text classification method using automatically evolved representations.
• To present autoBOT (automatic Bags-Of-Tokens), an automated machine learning (autoML) approach for resource-constrained scenarios.
• To enable competitive performance without the need for massive datasets or extensive computational resources.

Main Methods:

• An evolutionary algorithm jointly optimizes sparse text representations (word, subword, POS tags, keywords, knowledge graphs, relational features) and document embeddings.
• Evolution is performed at the representation level, not the learner level.
• The approach is designed for automated hyperparameter tuning in low-resource settings.

Main Results:

• autoBOT demonstrates competitive text classification performance across fourteen real-world tasks.
• The method achieves comparable or superior results to state-of-the-art models like BERT and RoBERTa.
• Performance is competitive against another autoML approach that evolves ensemble models.

Conclusions:

• autoBOT provides an effective and explainable solution for text classification in low-resource scenarios.
• The method reduces the dependency on large datasets and extensive computational resources.
• The explainability of autoBOT, stemming from input space importance, offers potential for meta-transfer learning.