Algorithm Perception When Using Threat Intelligence in Vulnerability Risk Assessment

Recent government and commercial initiatives have pushed for the use of the automated, artificial intelligence (AI)-based, analysis of cyber threat intelligence. The potential bias that might be present when evaluating threat intelligence coming from human and AI sources has to be better understood before deploying automated solutions to production. We present a controlled experiment with $n=57$ master students who had a mix of experience in security and machine learning to measure the bias introduced by the source of intelligence (human vs. AI). Each participant analyzed eight threat intelligence reports from the Dutch National Cyber Security Center where the source of the final recommendation was manipulated as for coming from a human expert or an AI algorithm. Our findings revealed that participants tended to disagree with the recommendation when it was coming from AI. While expertise on ML did not have any impact, we found that participants with more security expertise tended to agree with the recommendation. In contrast, we found that the perceives bias was statistically equivalent (TOST) whether the recommendation was coming from a human or from an AI. The only (expected) factor which had an impact on perceived bias was when participants disagreed with the recommendation (irrespective whether it was human or AI). These results provide insight on the possible impact of introduction on AI on rank-and-file Tier 1 SOC analysts. The generalization of our results to professional practice requires more experiments with experienced security professionals.