A novel Data and Model Centric artificial intelligence based approach in developing high-performance Named Entity...

Area of Science:

• Natural Language Processing (NLP)
• Artificial Intelligence (AI)

Background:

• Named Entity Recognition (NER) is crucial for domain-specific NLP applications, but existing Bengali NER models lack accuracy due to linguistic complexity and resource scarcity.
• High-resource languages like English and Chinese have well-performing NER architectures, highlighting a gap for morphologically rich languages such as Bengali.

Purpose of the Study:

• To achieve state-of-the-art performance in Bengali Named Entity Recognition (NER) by combining Data-Centric and Model-Centric AI.
• To propose a method for developing high-quality NER datasets applicable to any language.
• To evaluate the impact of a high-quality dataset on NER model accuracy.

Main Methods:

• Developed a unique, high-quality dataset specifically for Bengali NER.
• Integrated Data-Centric AI (dataset quality) and Model-Centric AI (hybrid deep learning model) approaches.
• Evaluated various Deep Learning models using the created dataset.

Main Results:

• A hybrid model achieved an exact match F1 score of 87.50%, partial match F1 score of 92.31%, and micro F1 score of 98.32%.
• The study demonstrated the significant impact of a high-quality dataset on NER model performance.
• The proposed model minimizes the need for manual feature engineering and requires fewer computational resources.

Conclusions:

• The integrated AI approach and high-quality dataset significantly advance Bengali NER capabilities.
• The developed method for dataset creation is generalizable to other languages.
• This work offers a resource-efficient and accurate solution for NER in morphologically rich languages.