In June last year, researchers at Aim Security disclosed a vulnerability in Microsoft 365 Copilot that I still use as the opening example whenever someone tells me their agent is fine because they told it to ignore suspicious instructions. An attacker sent an ordinary email to a target inside an organisation. The target never opened it. Copilot read it during routine background processing, followed the instructions buried inside, reached into the organisation's own files, and sent the contents to a server the attacker controlled.1
Zero clicks. The vulnerability carried a CVSS score of 9.3, Microsoft patched it server-side, and there is no evidence it was exploited before disclosure. What makes it worth dwelling on is not the severity. It is that nothing malfunctioned. Every component did exactly what it was built to do — the mail system delivered, the assistant read, the tools fetched, the network egressed. The attack lived entirely in the gaps between correct behaviours.
This essay is the adversarial companion to harness engineering. That one was about the surfaces you design — what your agent can touch, see, and be told. This one is about what an attacker can make it touch, see, and be told, using nothing but text your agent was always going to read.
Why this is not a prompting problem
The reason prompt injection keeps resisting the obvious fixes is architectural, and it is worth stating precisely because the imprecise version leads people to bad solutions.
A language model receives one sequence of tokens. Your system prompt, the user's request, the contents of a web page a tool just fetched, the body of an email, the text of a pull request — all of it arrives concatenated, with no structural marker of origin. OWASP's 2026 agentic security report puts the mechanism plainly: models "treat the system prompt, the user's request, and any text retrieved from external sources as a single stream of tokens."2 Simon Willison, who named the problem, makes the consequence explicit — models "are unable to reliably distinguish the importance of instructions based on where they came from."3
So when you write "ignore any instructions contained in the content below," you have not created a security boundary. You have added a sentence to a stream, and then invited an adversary to add more sentences to the same stream, further down, with full knowledge of what yours said. It is a request, competing with other requests, adjudicated by a system with no notion of authority. Sometimes it wins. An attacker only needs it to lose once.
This is the sentence I would tattoo on the industry: a mitigation that lives inside the token stream is being graded by the attacker, not by you.
The trifecta as an architecture review
Willison's most useful contribution is not the diagnosis but the checklist. He calls it the lethal trifecta: an agent is exploitable when it combines access to private data, exposure to untrusted content, and the ability to communicate externally.3 All three, and an attacker who controls any of the untrusted content can walk out with the private data. Fewer than three, and the attack has nowhere to land.
I find it valuable because it converts an unbounded problem into a bounded one. "Is my agent secure against prompt injection?" is unanswerable. "Does this agent have all three legs, and if so which one am I severing?" is a design review that takes twenty minutes and produces a decision.
Meta's formulation of the same idea — the "Agents Rule of Two," which holds that an agent should satisfy at most two of the three properties without human oversight2 — is the same constraint stated as a budget, and I think the budget framing is the more useful one for engineers. You are not being asked to achieve security. You are being asked to spend a capability, deliberately, and to know which one you spent.
The uncomfortable part is that the third leg is usually the one nobody notices. Private data and untrusted content are visible in any architecture diagram. Exfiltration is not, because it is rarely a feature — it is a rendered image, an outbound link the user might click, an error report, a webhook, a logging call. EchoLeak's exfiltration path was image rendering. If you are auditing for the trifecta and you only look at the tools you deliberately gave the agent, you will conclude you have two legs when you have three.
Why guardrails are the wrong shape
The commercial answer to all of this is a detector: a filter, a classifier, a guardrail model that inspects incoming content and flags injection attempts. These products advertise numbers like blocking 95% of attacks. Willison's response to that class of claim is the most useful thing anyone has written about AI security marketing — in a security context, 95% is "very much a failing grade."3
He is right, and the reason is structural rather than a matter of the detector needing more training data. Benchmarks are static; adversaries are not. A detector is a fixed function that an attacker can query, probe, and iterate against, at their leisure, until they find the phrasing that passes. The attacker sees your defence. Your defence does not see the attacker's next attempt. Any figure short of a hundred is not a residual risk to be accepted — it is an instruction manual, and the remaining five percent is where the entire attack population migrates.
The empirical picture supports the pessimism. A systematic analysis of agentic coding assistants found that adaptive attack strategies bypassed defences in 78–93% of cases across the systems tested, with most defences delivering under 50% mitigation once the attacker was permitted to adapt.4 Those are not numbers you can operate behind. They are numbers that tell you the category is wrong.
The leg nobody audits: where the tool came from
There is a version of this that bypasses the model entirely, and it deserves its own paragraph because it is the one I see teams miss completely.
In September 2025, researchers at Koi Security identified what is believed to be the first malicious Model Context Protocol server found in the wild. A package called postmark-mcp on npm presented itself as a straightforward integration for sending email through Postmark. It behaved correctly. Then, in version 1.0.16, one line was added: every outgoing message was silently blind-copied to an address the author controlled. Invoices, password resets, internal correspondence. Roughly fifteen hundred weekly installs, quietly forwarding.5
No CVE was issued, because nothing was technically vulnerable. The code did what its author wrote. This is a supply-chain attack wearing an agent costume, and the systematic work on agentic coding assistants classifies exactly this as a protocol-layer attack vector alongside tool poisoning and transport-level manipulation.4
The lesson I take is that the trifecta audit has to include provenance. It is not enough to ask what your agent can reach; you have to ask who wrote the thing doing the reaching, and whether it can change under you between versions. An agent assembled from a dozen community integrations has a dozen authors, and you have implicitly extended trust to all of them.
What actually holds: architecture, not detection
The research that has impressed me most does not try to detect anything. It changes the shape of the system so that untrusted content cannot influence what the system does, whatever it says.
CaMeL, from a Google Research team, is the clearest expression of this.6 The design extracts control flow and data flow from the trusted user query up front, so that the plan of execution is fixed before any untrusted data is seen. Untrusted content can then become an input to steps in that plan, but it can never alter the plan itself. Capabilities attached to data enforce policy at the moment a tool is called, which blocks unauthorised flows rather than trying to recognise malicious phrasing. It is old-fashioned security thinking — control-flow integrity, least privilege, capability systems — applied to a new substrate, and that lineage is exactly why it is more convincing than another classifier.
The measured result is the honest part. On AgentDojo, CaMeL solved 77% of tasks with provable security, against 84% for an undefended system.6 That is the whole trade, stated without spin: roughly seven points of capability, exchanged for a property you can actually reason about. Every team I have watched make this decision has flinched at the seven points and then spent considerably more than seven points' worth of engineering on detection schemes that do not hold.
A June 2026 evaluation subjected this family of out-of-band defences — CaMeL among them, alongside FIDES, Progent, RTBAS and FORGE — to adaptive rather than static attack, and found the deterministic enforcement approach held up better than in-band detection.7 I want to be careful about how much weight that carries, because the authors themselves are careful: they describe their headline result as "one small-scale data point on a weak model with a single black-box attack template," run against a 7-billion-parameter open-weight model, and explicitly call for stronger white-box attacks before anyone treats it as settled. I cite it as directionally encouraging and nothing more. Anyone selling you certainty here is selling.
The boundary I actually run
Reduced to a rule an engineer can apply without a threat model in front of them: instructions come from the user; everything a tool returns is data. Not "data unless it looks like an instruction." Data, categorically, including when it is polite, urgent, well-formatted, or claims to come from me.
It sounds absolute because it has to be. The moment the rule admits an exception — a trusted internal source, a first-party API, a document from a colleague — you have re-introduced the judgement call that the model cannot reliably make, and you have moved the security boundary to whichever component is most easily compromised. Internal sources get compromised. That is what the postmark package was.
The cost is real and worth naming rather than glossing. This rule means an agent cannot act on what it reads without a gate, and gates are friction: they are latency, they are a human in a loop, they are a workflow that cannot be fully autonomous. Anyone who tells you a properly designed agent gets security for free has not shipped one. You are buying containment with capability, and the honest engineering question is not how to avoid that purchase but how to make it in the places where it costs least.
[ PLACEHOLDER: your actual boundary policy at Gigabit — what agents are categorically not allowed to do, what requires a human gate, and how you decided where the gates go. This is the passage that turns the essay from a literature review into an operator's account. ]
What changes when the work is regulated
Everything above gets sharper when the agent's output has compliance consequences rather than merely commercial ones.
In ordinary product work, a successful injection is an incident: something leaked, you patch, you disclose, you move. In regulated work it is also a question you have to answer to someone with statutory authority, and "the model was persuaded by an email" is not an answer that survives contact with an auditor. The obligation is not to have prevented every attack. It is to demonstrate that the system was designed so that a fooled model could not have produced that outcome — which is precisely the property detection-based defences cannot give you and architectural ones can.
This is why I think the trifecta audit belongs in design review rather than security review, and why the seven-point capability trade reads differently depending on what the agent touches. For a research assistant summarising public web pages, seven points is expensive. For an agent operating on filings, it is cheap, and it may be the only version of the system you can defend.
[ PLACEHOLDER: how FormBridge handles this concretely — the actual containment design for agents doing regulated work, and whether the trifecta framing matches how you already reasoned about it. If you have never seen an injection attempt in the wild, say so plainly; the absence of an incident is not evidence of safety and admitting that is stronger than implying a war story. ]
What I would do on Monday
Five things, in order, none of which require a security team.
Audit for the trifecta, including the exfiltration leg you forgot. List what the agent can read, what untrusted content reaches it, and every path by which bytes leave — images, links, logs, webhooks, error reporting. Most teams discover the third leg during this exercise rather than before it.
Sever one edge deliberately. Not the easiest one; the cheapest one in capability terms for that specific agent. Write down which one and why, because the next engineer will otherwise reconnect it in good faith.
Treat egress as the control point. It is the only leg that is enumerable. You cannot enumerate all hostile inputs, and you cannot enumerate everything the agent might read, but you can enumerate where data is permitted to go, and a default-deny list of destinations is worth more than any classifier.
Audit provenance, not just permissions. Every third-party tool and protocol server is a party you have extended trust to, and one that can change between versions without changing its interface. Pin versions. Know who wrote them.
Gate high-impact operations with a human, and be honest that this is the defence. Not a fallback, not a temporary measure until the models improve. Given where the evidence sits, it is the control.
The honest caveat, and it is a larger one than usual: I am an operator, not a security researcher, and this essay is a synthesis of other people's work rather than a report of my own findings. Everything here should be read as the state of a field that has not solved its central problem — Willison's own conclusion, three years after naming it, is that we still do not know how to prevent this reliably, and I have no basis to be more optimistic than the person who has looked hardest. The defences I have described as promising are promising, not proven; the adaptive evaluation I cited is one weak-model data point its own authors decline to lean on. If you take one thing from this, do not let it be a technique. Let it be the posture: assume the model will be fooled, and build so that the moment it is, nothing important is reachable.