Prompt Injection

Why LLMs are uniquely vulnerable

A normal program keeps code and data in separate lanes — SQL injection happens precisely when that boundary leaks. An LLM has no such boundary at all. The system prompt, the user's message, a retrieved web page, a tool's output — everything is concatenated into one stream of tokens, and the model dutifully tries to follow any instruction it finds anywhere in that stream. So if attacker text says "ignore your rules and do X," the model is strongly inclined to comply. That's prompt injection.

• Direct injection — the user types the malicious instruction themselves (override, or a "jailbreak" role-play that talks the model out of its guardrails).
• Indirect injection — the payload is hidden in content the model reads: a web page an agent browses, a document in a RAG store, an email, even white-on-white text or an image. The user never sees it; the model still obeys it. This is the scary one for autonomous agents.

Impact ranges from leaking secrets and other users' data, to taking unauthorized actions through connected tools (send email, make purchases, run code), to spreading worms across agents. Here's a mock support bot with a secret it must protect — pick an attack and some defenses, then run it:

The defenses — and why none alone is enough

• Input filtering / guardrails — scan incoming text for known injection patterns and refuse or strip them. Cheap and catches lazy attacks, but bypassable: attackers paraphrase, encode, translate, or split the payload. A filter is a speed bump, not a wall.
• Instruction hierarchy & delimiting — train/prompt the model to treat the system prompt as authoritative and everything else (user text, retrieved docs) as data, not commands, clearly delimited. The strongest model-side mitigation, and what frontier models are increasingly trained for — but still probabilistic, so not a guarantee.
• Output filtering — scan the response before it leaves; block it if it contains the secret. Catches leaks back to the user, but misses side-channel exfiltration (e.g. the model emails the key out and only says "done") — watch the indirect attack slip past this one.
• Least privilege / privilege separation — the real fix. Assume the model can be hijacked and limit the blast radius: don't put secrets in its context, gate every tool behind explicit authorization, require human approval for risky actions, separate trusted and untrusted data planes. If the bot never has the key or the email tool, there's nothing to steal.

The lesson is defense in depth: layer the cheap filters for the easy stuff, but architect as if every prompt is potentially adversarial. You can't fully "prompt your way" to safety — you contain the damage.