From model manipulation to the exfiltration of sensitive data, the rise of autonomous AI agents is expanding the attack surface for organizations. While the technology is still emerging, companies must already begin putting preventive safeguards in place.

From generative to agentic AI

One buzzword replaces another. After the frenzy around generative AI comes agentic AI—its natural evolution, but with exponentially greater capabilities. While large language models (LLMs) merely respond to prompts, agentic AI systems can plan, reason, and execute tasks autonomously, with little or no human oversight.

The productivity promise is huge: fleets of AI agents coordinating to handle complex workflows. Imagine an onboarding journey where agents automatically manage HR paperwork, IT account provisioning, training modules, and even access rights configuration—without a human ever stepping in.

The technology is still in its infancy, but adoption is accelerating. According to Deloitte, a quarter of companies using generative AI will launch pilots or proofs of concept for agentic AI this year, a figure expected to double by 2027. Gartner forecasts that by 2028, one-third of enterprise applications will embed agentic AI, compared to less than 1% in 2024—automating up to 15% of daily business decisions. Unsurprisingly, major B2B software vendors like Salesforce, ServiceNow, and Workday are already baking agentic AI into their portfolios.

When an agent steps out of line

New technology inevitably brings new risks. Agentic AI inherits all the vulnerabilities of generative AI—most notably prompt injection—but autonomy amplifies the danger. With a malicious instruction hidden in an image or a string of invisible characters, an attacker could redirect an agent to perform harmful or unauthorized actions, like shutting down servers.

“The fallout can be serious,” warns David Kopp, cybersecurity expert at Trend Micro. “Attackers could delete databases, alter workflows, or manipulate recruiting software. Imagine a malicious resume designed to smuggle in instructions that the AI executes silently, while the recruiter sees nothing unusual.”

Compromised agents could embed dormant malware payloads, siphon sensitive data, or trigger actions without any user involvement—rendering traditional phishing campaigns almost obsolete. Data poisoning attacks could inject false information into outputs, while “zombie agents” might overwhelm systems in a denial-of-service attack.

Machines elevated to admin status

The core challenge lies in identity and access management. Agentic AI agents act as “virtual employees,” connecting to IT systems and cloud services through APIs. But unlike humans, they multiply at scale: according to CyberArk’s 2025 Identity Security Landscape, machine identities already outnumber human ones by 82 to 1. Agentic AI will only accelerate this imbalance.

To be effective, some agents request elevated privileges—sometimes even system administrator rights. That creates a major risk of privilege sprawl.

“The principle should be the same as onboarding a new hire,” says Pierre Codis, AVP of Sales at Keyfactor. “Define the agent’s role clearly and assign only the access rights required. For example, an agent with access to email should only be able to contact a pre-approved recipient list.”

Jean-Christophe Vitu, VP Solution Engineers EMEA at CyberArk, adds: “We need to apply the same identity governance principles as for humans, but at much higher scale. With just-in-time access, agents receive only the necessary permissions at the exact moment they need them—limiting what can be exfiltrated if compromised.”

Vendors are also proposing frameworks. Trend Micro recommends a LEAN architecture to counter prompt injection, enforce data integrity and confidentiality, and limit hallucination risks. MITRE has launched ATLAS (Adversarial Threat Landscape for AI Systems), a knowledge base extending its ATT&CK framework to LLMs and agentic systems.

Standards, protocols, and governance gaps

Zero Trust principles—least privilege, strong authentication, and continuous monitoring—apply neatly to agentic AI. In this model, agents are treated like any untrusted machine or microservice. PKI-based identity (digital certificates such as X.509) and Certificate Lifecycle Management (CLM) tools can reduce the risk of privilege misuse or expired credentials.

Yet the technology lacks formal standards. Two emerging protocols could change that:

  • Model Context Protocol (MCP), launched by Anthropic in late 2024, aims to provide a universal way to connect agents with enterprise apps, databases, and cloud services.
  • Agent2Agent (A2A), developed by Google, enables communication and orchestration between agents themselves.

But neither currently embeds robust security by design.

Continuous monitoring is non-negotiable. “Agents must be audited, logged, and correlated in real time,” stresses Vitu. “If an agent strays from its defined role, it should be blocked and isolated immediately.” Organizations must also define “forbidden actions” based on their specific operational risks.

Governance remains another weak spot. “Often IT teams spin up AI agents without involving security,” warns Trend Micro’s David Kopp. “Security teams then step in late in the cycle, forced to rework code before deployment. It risks repeating the early mistakes of cloud adoption—before DevSecOps became the norm.”

Machine versus machine

If businesses see agentic AI as a productivity tool, attackers are already weaponizing it. By deploying their own swarms of malicious agents, they can automate entire kill chains—from reconnaissance to vulnerability scanning to customized phishing campaigns.

“Agentic AI enables large-scale, fully orchestrated attacks,” says Blandine Delaporte, Senior Director of Solutions Engineering at SentinelOne. “Attackers no longer need to manually select targets. Agents can find them, adapt the plan, and launch exploits autonomously.”

And when detected by defensive AI, malicious agents can adapt in real time—changing tactics, bypassing defenses, or escalating privileges. It truly becomes machine versus machine.

That’s why defenders are also moving fast. In April, SentinelOne introduced Purple AI Athena, an agentic AI designed to autonomously reason and act, moving toward a self-operating SOC. More recently, Trend Micro unveiled Agentic SIEM, positioned to tackle long-standing challenges in security event management through autonomous orchestration.

The bottom line

Agentic AI promises efficiency gains but also expands the attack surface dramatically. As adoption ramps up, CISOs face a dual challenge: prevent agents from turning rogue inside their own organizations, while preparing for a future where attackers wield equally autonomous adversaries.

The message from experts is clear: the time to set guardrails is now.

Stay tuned in real time
Subscribe to
the newsletter
By providing your email address you agree to receive the Incyber newsletter and you have read our privacy policy. You can unsubscribe at any time by clicking on the unsubscribe link in all our emails.
Stay tuned in real time
Subscribe to
the newsletter
By providing your email address you agree to receive the Incyber newsletter and you have read our privacy policy. You can unsubscribe at any time by clicking on the unsubscribe link in all our emails.