technical-term · Context & knowledge · foundational · Reviewed

Context window

The finite sequence of tokens a model can directly condition on during a single inference operation.

Definition

The context window contains the active instructions, user input, selected history, retrieved knowledge, tool results, and other state presented to the model. A larger window increases capacity but does not ensure relevance, consistency, truth, or correct attention.

Finite model context is established; effective use remains workload- and model-dependent.

Why it matters

Context is a scarce runtime resource even when nominal token limits are large.

Selection, ordering, provenance, compression, and removal are engineering decisions.

System anatomy

Capacity
The maximum token budget available to the inference.
Composition
The mix and ordering of instructions, state, examples, evidence, and retrieved data.
Provenance
Where each context item came from and what trust level it carries.

Important distinctions

Memory
Memory persists or retrieves information across time; context is what is visible for this decision.
Knowledge
Presence in context does not make a statement true or authoritative.

Implementation signals

  • Load the smallest sufficient context for the decision
  • Label source, recency, and trust level
  • Measure performance under realistic context length and noise

Failure modes

  • Context stuffing hides the governing requirement
  • Stale summaries silently replace primary evidence

Sources and further study

  1. Anthropic — Effective context engineering for AI agents

    A working model for treating model-visible context as a finite resource that must be selected and maintained.

    Use in this library: First-party case study. This first-party account documents one organization, product, or implementation context and should not be generalized without local evidence.

    engineering · engineering · Published 2025-09-29
  2. Peter Yang — 5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

    A Linear engineering discussion of production agents, tool design, context loading, feedback, evaluation, and simple model-in-a-loop explanations.

    Use in this library: Practitioner perspective. The five rules and “LLM in a loop plus tools” shorthand reflect one product team’s experience; they are not a standard or a complete enterprise-agent definition.

    video · video · Published 2026-08-09