The enforcement map

Governance documents say things like "the agent must never refund above $200." The architect's job is the next sentence, the one the document usually skips: enforced where? Every policy has four candidate homes β€” the retrieval scope, the tool scope, the output screen, the approval gate β€” and each home stops a different failure class. Putting a policy in the wrong home is how systems end up governed on paper and ungoverned in production.

enforcement pointtool scope approval gateoutput screen

One policy, three homes, measured

The policy: refunds are capped at $200, and must hit the right order. 1,000 refund requests for $20–$400, of which 2% target the wrong order entirely (the customer pasted the wrong id; the model matched the wrong record). Choose where the cap lives:

Where does the refund cap live?

Read the three configurations as a progression of failure classes. The prompt version leaks over-cap refunds β€” not many, but "not many" times real money is an incident report. The schema version makes an over-cap refund unrepresentable: the tool call cannot validate, so the failure class is closed by construction β€” but the schema knows nothing about which order deserves the refund, so wrong-target refunds sail through at exactly the background rate. The gate closes that last class too, at the price every gate charges: human time, on every single approval.

The map itself

Generalise that experiment and you get the table the exam keeps asking about from different angles β€” each policy type has a home where it's enforceable by construction, and prompt-level phrasing of the same policy is a behaviour shaper, never the control:

policyenforced by construction at…the prompt version fails when…
"never reveal internal pricing"retrieval scope β€” pricing docs aren't retrievable in this flow (see ACL-RAG)the model is argued into it, or paraphrases what's in context
"refunds ≀ $200"tool scope β€” maximum: 200 in the schema; larger calls cannot validatean urgent-sounding customer justifies an exception
"no destructive action without sign-off"approval gate β€” the harness holds the call until a human approvesthe model decides this case surely counts as approved
"no PII in replies"output screen (recognisable formats: emails, card numbers) β€” plus retrieval scope for PII that shouldn't be in context at allthe PII is paraphrased, partial, or a format the screen doesn't know
"stay under $0.25/episode"harness budget β€” the loop is cut when the meter says so (see the Bench loop station)always β€” the model doesn't see the meter

Two working rules fall out. First: when a policy review says "the prompt says so," ask what happens on the day the prompt is ignored β€” if the answer is "an incident," the policy needs a structural home too. Second: gates are precious β€” they spend human attention, the scarcest resource in the system β€” so they belong on the actions that are irreversible or above authority, not on everything (a gate on everything is how reviewers learn to click approve without reading).

⚠️ Traps & honesty: the simulation's 7% instruction-ignore rate and 2% wrong-target rate are assumptions of the toy β€” the structure (each home closes exactly one failure class) is the transferable part Β· "enforced by construction" is only as good as the construction: a schema cap of 200 enforces nothing if a second, uncapped refund tool exists Β· output screens are genuinely useful for fixed-format PII and genuinely weak against paraphrase β€” deploy them knowing which of the two you're getting Β· the prompt still matters: it prevents most attempts from ever reaching the structural control, which is what keeps escalation queues short.
Takeaways: every policy needs an answer to "enforced where?" β€” the four homes are retrieval scope, tool scope, approval gate, output screen (plus harness budgets for money and turns) Β· a prompt phrasing of a policy shapes behaviour and cannot guarantee it β€” pair it with the structural home Β· schemas close the bounds failure class; gates close the judgment failure class; neither substitutes for the other Β· spend gates only where actions are irreversible or above authority β€” approval fatigue is itself a risk. Next: GDPR, HIPAA & FedRAMP.

Second opinion (taught here β€” these corroborate): Building effective agents Β· Anthropic docs Β· tool use.