# Private AI An operating posture in which an organization defines and enforces where AI data, models, inference, tools, logs, identities, and administrative access may exist and flow. technical-term · Security & governance · contested · Reviewed 2026-08-09 ## Definition Private AI is a deployment and data-governance posture, not a model type. It combines an explicit threat model with controlled infrastructure, data boundaries, model and tool access, logging, retention, administrative controls, and evidence. Local or self-hosted inference can support that posture, but neither proves privacy when prompts, telemetry, dependencies, operators, or tool calls cross the intended boundary. The underlying privacy and infrastructure controls are established, while “private AI” remains a broad market term with inconsistent boundaries and evidence. ## Why it matters A useful claim names the protected data, permitted processors, geographic and network boundaries, operators, retention, and exceptions. Privacy must cover the full execution path: retrieval, prompts, model runtime, caches, tools, traces, backups, support access, and model updates. ## System anatomy - **Threat model:** The actors, data, actions, and failure paths the posture addresses. - **Processing boundary:** Where data and model execution are permitted to occur. - **Access control:** Who or what may administer, invoke, inspect, export, or update the system. - **Evidence:** Configuration and operating records supporting the stated privacy claim. ## Important distinctions - **Local inference:** Locality describes where one runtime executes; privacy covers the complete data and control path. - **Open-weight model:** Downloadable weights can enable deployment choice but do not determine privacy, licensing, or security. - **Air-gapped deployment:** Network isolation is one possible control, not the definition of private AI. ## Implementation signals - Write a data-flow and operator-access threat model - Disable or constrain cloud features when the intended boundary requires it - Verify logs, caches, backups, tool traffic, and support paths - State precisely which privacy properties are implemented ## Failure modes - A local model sends tool requests or telemetry outside the boundary - Prompts remain in ungoverned logs or backups - The label “private” substitutes for testable controls ## Related knowledge - [Local inference](https://aisdlc.ai/agentic-engineering/local-inference) — Model inference executed on the user’s or organization’s local hardware and runtime rather than delegated to a separately operated remote model endpoint. - [Self-hosted inference](https://aisdlc.ai/agentic-engineering/self-hosted-inference) — An inference service whose runtime, capacity, configuration, model artifacts, and operational controls are managed by the organization using it or by a directly governed operator. - [Open-weight model](https://aisdlc.ai/agentic-engineering/open-weight-model) — A model whose trained parameter weights are available for download and use under stated license terms, enabling inspection, adaptation, or deployment outside a hosted API. - [Instruction–data trust boundary](https://aisdlc.ai/agentic-engineering/instruction-data-trust-boundary) — An architecture that distinguishes authoritative instructions from retrieved content, memory, tool results, and external data through provenance, trust labels, privilege separation, validation, and mediated action. ## Sources and further study - [Ollama — Ollama FAQ](https://docs.ollama.com/faq) — Documents local prompt handling, local-only mode, networking, storage, and runtime configuration for operating models through Ollama on user-controlled hardware. **Use:** Official guidance. **Limitation:** Vendor documentation describes runtime behavior and configuration. Local execution alone does not secure the host, logs, network, tools, model license, or governance process. - [ggml.org — llama.cpp](https://github.com/ggml-org/llama.cpp) — Documents a C and C++ runtime for large-model inference across a wide range of local and cloud hardware, including GGUF models, quantization support, and an API server. **Use:** First-party case study. **Limitation:** This is implementation documentation for an inference runtime, not a complete agent, security boundary, governance system, or guarantee of model quality and performance. - [NIST — Zero Trust Architecture (NIST SP 800-207)](https://csrc.nist.gov/pubs/sp/800/207/final) — Defines zero trust around explicit, continuously evaluated access decisions rather than implicit trust from network location. **Use:** Standard or protocol. **Limitation:** The publication is an enterprise security architecture, not an agent-specific verification standard; agent applications require additional threat and assurance analysis. --- This library synthesizes cited research, standards, official documentation, and clearly attributed practitioner perspectives. Maturity describes the state of a concept—not vendor endorsement, production readiness, or permission to deploy. Benchmarks and demonstrations do not replace use-case evaluation, governed controls, independent verification, or named human release authority.