Question classification based on Bloom's taxonomy cognitive domain using modified TF-IDF and word2vec

Area of Science:

• Educational Technology
• Natural Language Processing
• Artificial Intelligence in Education

Background:

• Effective student assessment is vital in educational settings, with examinations being a primary evaluation tool.
• Bloom's taxonomy is a widely adopted framework for categorizing cognitive skills in educational questions.
• Existing automated question classification methods often focus on single domains, lacking multi-domain applicability.

Purpose of the Study:

• To develop and present a novel classification model for automatically categorizing exam questions based on Bloom's taxonomy across multiple domains.
• To address the limitation of existing methods that primarily classify questions within specific subject areas.

Main Methods:

• Proposed a hybrid feature extraction technique combining Term Frequency-Part of Speech-Inverse Document Frequency (TFPOS-IDF) and pre-trained word2vec embeddings.
• TFPOS-IDF was utilized to assign appropriate weights to significant terms within questions based on their part of speech.
• The combined features were input into three distinct classifiers: K-Nearest Neighbour, Logistic Regression, and Support Vector Machine.

Main Results:

• The model demonstrated strong performance on two datasets (141 and 600 questions).
• Weighted F1-measures achieved were 71.1%, 82.3%, and 83.7% for the first dataset.
• Weighted F1-measures for the second dataset reached 85.4%, 89.4%, and 89.7% respectively.

Conclusions:

• The proposed method effectively classifies multi-domain exam questions according to Bloom's taxonomy.
• The combination of TFPOS-IDF and word2vec features significantly enhances classification accuracy.
• This approach offers a valuable tool for improving the quality and balance of educational assessments.