Student assessment in cybersecurity training automated by pattern mining and clustering

Area of Science:

• Cybersecurity Education
• Educational Data Mining
• Machine Learning Applications

Background:

• Hands-on cybersecurity training offers practical skill development in interactive environments.
• Trainee interaction data, including command-line usage, can indicate learning processes.
• Automated analysis of this data is complex due to diverse problem-solving approaches and large data volumes.

Purpose of the Study:

• To explore the application of data mining and machine learning techniques for analyzing cybersecurity training data.
• To identify patterns in trainee behavior, mistakes, and learning challenges.
• To assess the suitability of these methods for improving cybersecurity training.

Main Methods:

• Utilized pattern mining and clustering techniques.
• Analyzed 8834 commands from 113 trainees across 18 cybersecurity training sessions.
• Focused on identifying typical behaviors, errors, solution strategies, and difficult training phases.

Main Results:

• Pattern mining effectively captured timing and tool usage frequency.
• Clustering revealed common issues faced by multiple trainees, suggesting a need for targeted support.
• The study demonstrated the feasibility of using data mining for analyzing cybersecurity training interactions.

Conclusions:

• Data mining and machine learning methods are suitable for analyzing complex cybersecurity training data.
• These methods can provide insights into trainee learning processes, enabling better assessment and feedback.
• Findings support the application of these techniques by educational researchers and practitioners to enhance training design and trainee support.