CompPhish: A comprehensive dataset for diversified phishing detection

Most of the cyber attacks are initiated through phishing URLs, which are shared with the victims through multiple media. In spite of the research community proposing varied solutions, the volume and nature of such attacks have evolved unpredictably. To develop effective solutions, researchers require comprehensive datasets that encompass a wide range of attack types rather than focusing on a narrow subset. We present CompPhish, a processed dataset, comprising of 15,358 URLs paired with their respective HTML sources. 7204 URLs are phishing, and 8154 are legitimate. A set of 70 features, specifically curated to capture the properties exhibited by phishing URLs, is extracted. These features represent a diverse range of phishing attacks, including generic URL-based phishing attacks, phishing through brand-jacking, phishing sites hosted on compromised domains, and auto-downloadable malicious files embedded in webpages. Phishing URLs are gathered from PhishTank and OpenPhish, while legitimate URLs are compiled from multiple independent sources, like DataforSEO and GitHub. By publishing the raw URLs and HTML codes along with the feature vectors, CompPhish attempts to aid the researchers in devising generalized, robust, and reliable machine learning based solutions for real-world phishing attempts.