Lightweight malicious URL detection using deep learning and large language models

With thousands of new websites emerging daily, distinguishing between legitimate and malicious web pages has become increasingly challenging, as many of these sites compromise users' private data without consent, posing severe cybersecurity threats. The absence of robust detection mechanisms exposes users to cyberattacks, financial fraud, and identity theft. While several Machine Learning (ML)-based techniques exist, they suffer from limitations such as reliance on handcrafted features and difficulty in adapting to evolving attack patterns. To mitigate these challenges, this paper introduces a fully automated deep learning (DL) based framework designed for the detection of malicious Uniform Resource Locators (URLs). The framework utilizes Large Language Models (LLMs) to generate high-quality URL embeddings that capture complex patterns and token relationships in URLs without manual feature engineering. These embeddings are then classified into four categories, i.e., defacement, malware, benign, and phishing, using a customized DL-based model that is finalized using extensive ablation experiments. The proposed DL model uses Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) layers to capture long-range dependencies between the embeddings. The proposed system achieved the highest accuracy of 97.5% using a Bidirectional Encoder Representations from Transformers (BERT) and a DL-based model. With only 0.5 M parameters, the BERT + DL model can classify samples in 0.119 ms. Additionally, to enhance interpretability and trustworthiness, the eXplainable AI (XAI) technique called Local Interpretable Model-Agnostic Explanations (LIME) is used to visualize model decisions to ensure the model's transparency and reliability in a real-time setting.