Sustainable science mapping: benchmarking green AI against transformers for cross-disciplinary abstract...

Area of Science:

• Computational linguistics
• Artificial Intelligence
• Bibliometrics

Background:

• Scholarly literature is growing exponentially, requiring automated systems for knowledge organization.
• Classifying academic abstracts is challenging due to specialized language and diverse disciplinary structures.

Purpose of the Study:

• To propose a resource-efficient deep learning method for categorizing academic abstracts.
• To compare the performance and computational efficiency of Recurrent Neural Networks (Attention-GRU) and Transformer architectures (BERT, SciBERT).

Main Methods:

• Developed and evaluated an Attention-based Gated Recurrent Unit (GRU) model with GloVe embeddings.
• Conducted comparative analysis against Transformer models (BERT, SciBERT) on large-scale datasets, including WOS-46985 (134 sub-disciplines).
• Focused on the trade-off between predictive accuracy (Macro-F1 score) and computational resources (training time, energy proxy).

Main Results:

• The Attention-GRU model achieved a Macro-F1 score of 0.920, surpassing SciBERT (F1: 0.867).
• Achieved over 3x faster training times compared to Transformer variants.
• Demonstrated significantly lower estimated energy consumption than Transformer models.

Conclusions:

• Computationally efficient models like Attention-GRU can effectively categorize fine-grained scientific taxonomies.
• This research supports "Green AI" by showing high performance is achievable without prohibitive energy costs.
• The proposed method offers a scalable and resource-efficient solution for organizing vast amounts of academic literature.