According to VentureBeat, Forrester’s 2025 Security and Risk Summit delivered some brutal reality checks about generative AI’s reliability and security implications. Research from Columbia University’s Tow Center found AI models are wrong 60% of the time across eight different systems including ChatGPT and Gemini. Carnegie Mellon’s AgentCompany benchmark showed failure rates soaring to 70-90% on complex corporate tasks, while Veracode’s 2025 GenAI Code Security Report revealed 45% of AI-generated code contains known OWASP Top 10 vulnerabilities. The identity management market is projected to surge to $27.5 billion by 2029 as organizations scramble to contain the chaos created by AI’s exponential multiplication of attack surfaces through machine identities.
The uncomfortable truth about AI reliability
Here’s the thing that nobody wants to admit: we’re deploying systems that fail more often than they succeed. When Carnegie Mellon researchers tested leading AI models against 175 real corporate tasks, the best performers completed only 24% autonomously. And that’s before adding complexity – then failure rates jump to 70-90%. Salesforce’s own research showed similar patterns, with CRM-oriented agents failing 62% of baseline enterprise tasks. But wait, it gets worse: when they added basic safety guardrails, accuracy dropped by half. Basically, the more we try to make AI safe, the worse it performs at actual work.
AI-generated code is a security disaster
The Veracode study really drives home how dangerous this reliability problem becomes when AI starts writing production code. 45% of AI-generated code containing known vulnerabilities isn’t just bad – it’s catastrophic. Java showed the worst results with only 28.5% security pass rate, while cross-site scripting and log injection vulnerabilities had pass rates of just 12-13%. The most alarming insight? Security performance remained flat even as models got better at generating syntactically correct code. Newer models produce more compilable code that’s still full of security holes. It’s like having a construction crew that builds beautiful buildings with structural flaws you can’t see until they collapse.
The identity explosion nobody prepared for
Now consider what happens when every AI system creates new machine identities at scale. Traditional identity governance simply can’t keep up. The recent OAuth token breach affecting 700+ Salesforce customers proved that API keys and certificates aren’t configuration artifacts – they’re high-value identities. When 88% of security leaders admit to using unauthorized AI in daily workflows, you’ve got shadow AI creating shadow identities everywhere. The $27.5 billion identity management market projection by 2029 tells you everything about the scale of this problem. Organizations are facing identity sprawl at machine speeds, and traditional security approaches are completely inadequate.
What security teams need to do now
So where does this leave us? Forrester’s presentation made it clear that AI red teaming needs to become standard practice. Traditional pentesting hunts infrastructure flaws, but AI red teaming simulates adversarial attacks on the models themselves. The challenge is that these systems fail in ways humans don’t – they hallucinate with absolute confidence, like placing shark attacks in landlocked Wyoming. When you combine 70-90% incompleteness with production deployment velocity, you’ve created the perfect conditions for security disasters. The bottom line: we need to stop treating AI as a magic solution and start treating it like the unreliable, dangerous tool it actually is.
