technical-term · Security & governance · contested · Reviewed

Private AI

An operating posture in which an organization defines and enforces where AI data, models, inference, tools, logs, identities, and administrative access may exist and flow.

Definition

Private AI is a deployment and data-governance posture, not a model type. It combines an explicit threat model with controlled infrastructure, data boundaries, model and tool access, logging, retention, administrative controls, and evidence. Local or self-hosted inference can support that posture, but neither proves privacy when prompts, telemetry, dependencies, operators, or tool calls cross the intended boundary.

The underlying privacy and infrastructure controls are established, while “private AI” remains a broad market term with inconsistent boundaries and evidence.

Why it matters

A useful claim names the protected data, permitted processors, geographic and network boundaries, operators, retention, and exceptions.

Privacy must cover the full execution path: retrieval, prompts, model runtime, caches, tools, traces, backups, support access, and model updates.

System anatomy

Threat model
The actors, data, actions, and failure paths the posture addresses.
Processing boundary
Where data and model execution are permitted to occur.
Access control
Who or what may administer, invoke, inspect, export, or update the system.
Evidence
Configuration and operating records supporting the stated privacy claim.

Important distinctions

Local inference
Locality describes where one runtime executes; privacy covers the complete data and control path.
Open-weight model
Downloadable weights can enable deployment choice but do not determine privacy, licensing, or security.
Air-gapped deployment
Network isolation is one possible control, not the definition of private AI.

Implementation signals

  • Write a data-flow and operator-access threat model
  • Disable or constrain cloud features when the intended boundary requires it
  • Verify logs, caches, backups, tool traffic, and support paths
  • State precisely which privacy properties are implemented

Failure modes

  • A local model sends tool requests or telemetry outside the boundary
  • Prompts remain in ungoverned logs or backups
  • The label “private” substitutes for testable controls

Sources and further study

  1. Ollama — Ollama FAQ

    Documents local prompt handling, local-only mode, networking, storage, and runtime configuration for operating models through Ollama on user-controlled hardware.

    Use in this library: Official guidance. Vendor documentation describes runtime behavior and configuration. Local execution alone does not secure the host, logs, network, tools, model license, or governance process.

    guidance · guidance · Published 2026
  2. ggml.org — llama.cpp

    Documents a C and C++ runtime for large-model inference across a wide range of local and cloud hardware, including GGUF models, quantization support, and an API server.

    Use in this library: First-party case study. This is implementation documentation for an inference runtime, not a complete agent, security boundary, governance system, or guarantee of model quality and performance.

    engineering · engineering · Published 2026
  3. NIST — Zero Trust Architecture (NIST SP 800-207)

    Defines zero trust around explicit, continuously evaluated access decisions rather than implicit trust from network location.

    Use in this library: Standard or protocol. The publication is an enterprise security architecture, not an agent-specific verification standard; agent applications require additional threat and assurance analysis.

    standard · standard · Published 2020-08-11