AI + a16z

Why Social Engineering Now Works on Machines

Episode Summary

Ian Webster built PromptFoo after watching 200 million Discord users systematically dismantle his AI agent—now Fortune 10 companies pay him to break theirs before customers do. The "lethal trifecta" sounds academic until you realize it's already happening: untrusted input plus sensitive data plus an exfiltration channel equals the security incident that just cost a SaaS company its multi-tenancy guarantees. Webster's red-teaming agents don't use signatures—they have 30,000 conversations with your system, socially engineering their way past guardrails the same way a teenager with emojis convinced ChatGPT to leak data, except his tools find the vulnerability before your users become the pen testers.

Episode Notes

Ian Webster built PromptFoo after watching 200 million Discord users systematically dismantle his AI agent—now Fortune 10 companies pay him to break theirs before customers do. The "lethal trifecta" sounds academic until you realize it's already happening: untrusted input plus sensitive data plus an exfiltration channel equals the security incident that just cost a SaaS company its multi-tenancy guarantees. Webster's red-teaming agents don't use signatures—they have 30,000 conversations with your system, socially engineering their way past guardrails the same way a teenager with emojis convinced ChatGPT to leak data, except his tools find the vulnerability before your users become the pen testers.

Follow Ian Webster on X: https://x.com/iwebst

Follow Joel on LinkedIn: https://www.linkedin.com/in/3448827723723234/