Large Language Models for Synthetic Dataset Generation of Cybersecurity Indicators of Compromise.

Area of Science:

• Cybersecurity
• Artificial Intelligence
• Data Science

Background:

• High-quality labeled datasets with Indicators of Compromise (IoCs) are scarce, hindering the development of effective Cyber Threat Intelligence (CTI) predictive models.
• Social media platforms are increasingly targeted, necessitating robust methods for IoC classification in these environments.

Purpose of the Study:

• To fine-tune OpenAI's GPT-3.5 Large Language Model (LLM) for generating synthetic datasets that mimic real social media data.
• To incorporate domain-specific IoC knowledge into the LLM for enhanced synthetic data generation.
• To evaluate the performance of machine learning and deep learning models on these synthetic datasets.

Main Methods:

• Fine-tuning GPT-3.5 LLM on a curated social media dataset with IoC domain knowledge.
• Generating two synthetic datasets (4,000 and 12,000 instances).
• Evaluating four ML/DL models (DenseNN, Logistic Regression) on the generated datasets.

Main Results:

• On the 4,000-instance dataset, Dense Neural Network (DenseNN) achieved the highest accuracy at 77%.
• On the 12,000-instance dataset, Logistic Regression (LR) achieved the highest accuracy at 82%.
• The study demonstrates the efficacy of fine-tuned LLMs for synthetic data generation in CTI.

Conclusions:

• Fine-tuning LLMs with domain knowledge is a viable strategy for creating high-quality synthetic datasets.
• The generated datasets can improve IoC extraction and classification, offering a novel resource for cybersecurity.
• This approach addresses the data scarcity challenge in CTI research and applications.