Identification of key sentences in a text

Area of Science:

• Natural Language Processing
• Educational Technology
• Artificial Intelligence

Background:

• Human evaluation of student assessments is time-consuming and cognitively demanding.
• Automated evaluation models often lack accuracy, especially with lengthy or complex student responses.
• A need exists for tools to assist human evaluators by filtering relevant information.

Purpose of the Study:

• To introduce a semantic-aware model, the Key Sentence Identifier (KSI), as a proof of concept.
• To extract topic-relevant and coherent sentences from student responses.
• To assist human evaluators by reducing time and cognitive load.

Main Methods:

• Developed a dual-embedding framework integrating BERT and SBERT.
• Curated a dedicated dataset for training and evaluation.
• Employed ablation analysis to determine optimal model configuration (SFT BERT + SFT SBERT).

Main Results:

• The SFT BERT + SFT SBERT configuration achieved an F1 score of 86.01%, a significant improvement from 50.32%.
• The KSI model outperformed baseline methods and LoRA-based fine-tuning in identifying relevant sentences.
• Demonstrated consistent performance in extracting contextually relevant information.

Conclusions:

• The KSI model efficiently identifies key sentences in descriptive answers, aiding human evaluation.
• KSI reduces evaluation time and cognitive effort, potentially improving assessment quality and learning outcomes.
• The model assists human evaluation and can be integrated into future automated systems, aligning with UN Sustainable Development Goal 4.