“Human study on AI spear phishing campaigns” by Simon Lermen, Fred Heiding
11:14 Share

TL;DR: We ran a human subject study on whether language models can successfully spear-phish people. We use AI agents built from GPT-4o and Claude 3.5 Sonnet to search the web for available information on a target and use this for highly personalized phishing messages. We achieved a click-through rate of above 50% for our AI-generated phishing emails.

** Full paper: https://arxiv.org/abs/2412.00586** This post is intended to be a brief summary of the main findings, these are some key insights we gained:

AI spear-phishing is highly effective, receiving a click-through rate of more than 50%, significantly outperforming our control group. AI-spear phishing is also highly cost-efficient, reducing costs by up to 50 times compared to manual attacks. AI models are highly capable of gathering open-source intelligence. They produce accurate and useful profiles for 88% of targets. Only 4% of the generated profiles contained inaccurate information. Safety guardrails are not a [...]

Outline:

(00:30) Full paper:

(01:35) Abstract

(03:19) Method

(04:03) Results

(06:30) Automated intent detection

(07:57) The economics of AI-enhanced phishing

(08:36) Future Work

(09:57) Conclusion

The original text contained 1 image which was described by AI.

First published: January 3rd, 2025

Source: https://www.lesswrong.com/posts/GCHyDKfPXa5qsG2cP/human-study-on-ai-spear-phishing-campaigns)

    ---
    

Narrated by TYPE III AUDIO).

Images from the article: ))))) Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts), or another podcast app.