core-concept · Security & governance · emerging · Reviewed
Instruction–data trust boundary
An architecture that distinguishes authoritative instructions from retrieved content, memory, tool results, and external data through provenance, trust labels, privilege separation, validation, and mediated action.
Definition
An architecture that distinguishes authoritative instructions from retrieved content, memory, tool results, and external data through provenance, trust labels, privilege separation, validation, and mediated action. Untrusted data can look like instructions to a model; syntax must never decide authority.
Prompt injection remains unsolved; layered controls reduce consequence but cannot perfectly classify intent.
Why it matters
Untrusted data can look like instructions to a model; syntax must never decide authority.
The operating shift is from “All context treated as text” to “Provenance-aware context trust zones.”
System anatomy
- Practice 1
- Label origin, authority, and trust for model-visible content
- Practice 2
- Never let retrieved content grant tools or permissions
- Practice 3
- Isolate and independently validate actions induced by untrusted data
Important distinctions
- All context treated as text
- This concept moves the engineering system toward provenance-aware context trust zones.
Implementation signals
- Label origin, authority, and trust for model-visible content
- Never let retrieved content grant tools or permissions
- Isolate and independently validate actions induced by untrusted data
Failure modes
- Prompt-only filtering presented as a security boundary
- External content entering durable memory and later appearing trusted
Sources and further study
- OWASP GenAI Security Project — State of Agentic AI Security and Governance 2.01
A current threat and governance synthesis covering instruction-data confusion, containment gaps, autonomous action, and agentic incident readiness.
Use in this library: Official guidance. Guidance describes recommended practice; citation does not prove that a control is implemented or effective in a particular environment.
guidance · guidance · Published 2026-06-01 - OWASP GenAI Security Project — OWASP Top 10 for Agentic Applications 2026
A threat-oriented reference for goal hijacking, tool misuse, identity abuse, memory poisoning, and cascading agent failures.
Use in this library: Official guidance. Guidance describes recommended practice; citation does not prove that a control is implemented or effective in a particular environment.
guidance · guidance · Published 2025-12 - Anthropic — Effective context engineering for AI agents
A working model for treating model-visible context as a finite resource that must be selected and maintained.
Use in this library: First-party case study. This first-party account documents one organization, product, or implementation context and should not be generalized without local evidence.
engineering · engineering · Published 2025-09-29