Private AI and owned inference

01 · Field module

Trace where the data comes to rest

Where does your data come to rest once a prompt leaves the application?

Trace every place prompts, retrieved context, memory, tool arguments, telemetry, backups, identities, and operator access carry data across a boundary.

Learning objectives

• Follow one request from user input through the model call, tool calls, traces, logs, and backups.
• Separate the claim that weights run on your hardware from the claim that data stays inside a boundary.
• Name every identity and operator that can read each store the request touches.
• Record confirmed retention periods and mark the rest as unknown.

Study sequence

1. What private AI claims and what it does not · read · 16 min

   Separate where weights run from where data goes.

2. Compare the stores a single request creates · compare · 20 min

   Set memory, traces, and backups side by side before mapping.

3. Trace one request end to end · practice · 30 min

   Produce the data-path map later decisions are argued against.

Field exercise

Trace one workload you already run and record every place its data comes to rest or crosses an operator boundary.

Deliverable: A data-path map for one workload with a store-by-store access and retention register.

Independent challenge: A reviewer who did not build the map picks two stores at random and asks the learner to show the configuration that proves the recorded access and retention.

Evidence to retain

• Data-path map
• Store register
• Identity and operator access list
• Retention notes separating confirmed from unknown
• Unresolved question list

02 · Field module

Choose how much of the stack to own

How much of the inference stack do you need to own to satisfy the constraint you actually have?

Compare hosted APIs, dedicated endpoints, self-hosted open weights, adapted models, and pretraining against a written constraint rather than a preference.

Learning objectives

• State the binding constraint as residency, contractual control, latency, cost, or capability before comparing options.
• Compare the ownership levels on what each one changes and what each one leaves unchanged.
• Record the operating burden each level transfers onto your own team.
• Name the alternative you rejected and the evidence that rejected it.
• State the conditions under which pretraining or continued pretraining would be considered, and why they do not apply to this workload.

Study sequence

1. What open weights change · read · 14 min

   Establish what possession of weights does and does not grant.

2. Compare the ownership levels side by side · compare · 22 min

   Force each ownership level to answer the same questions.

3. Write the decision against the constraint · practice · 30 min

   Convert the comparison into a record another person can challenge.

Field exercise

Write a one-page ownership decision for a real workload and defend it against the adjacent ownership levels in both directions.

Deliverable: An ownership decision record for one workload with the binding constraint, the comparison, and the rejected alternatives.

Independent challenge: A reviewer argues for the adjacent level — the cheaper one below the choice, or the higher-ownership one when the lowest level was chosen — and the learner must answer with the binding constraint and the recorded figures rather than with preference.

Evidence to retain

• Constraint statement
• Option comparison table
• Operating-burden note
• Rejected-option rationale
• Named decision owner

03 · Field module

Run inference on hardware you control

What does a model actually need from your hardware before it can carry a real workload?

Match model format, quantization, memory, context length, and latency to a workload you measure on hardware you control.

Learning objectives

• Relate parameter count, quantization format, and context length to a memory footprint.
• Measure first-token latency and sustained throughput on your own machine instead of quoting published figures.
• Identify which of your workloads a local model can carry and which it cannot.
• Record the quality difference between quantization levels, or record that you did not measure it.

Study sequence

1. Local inference and what it costs you · read · 15 min

   Name the resource limits before touching a model.

2. Compare quantization levels of one model · compare · 20 min

   Separate the memory saving from the quality cost.

3. Measure your own hardware · practice · 35 min

   Replace published benchmarks with figures from the target machine.

Field exercise

Run one open-weight model on hardware you control and record what it can and cannot carry for a real task of yours. Do this even where the chosen ownership level is hosted — the measurement is what makes that choice defensible rather than assumed.

Deliverable: A hardware fit report for one model and one task, with measured latency, memory headroom, and quality observations.

Independent challenge: A reviewer reruns one recorded configuration on the same machine and checks whether the numbers reproduce within the tolerance the learner stated.

Evidence to retain

• Model and quantization identifiers
• Hardware specification
• Latency measurements
• Memory-headroom record
• Quality observation notes

04 · Field module

Serve it like production infrastructure

What breaks once a model you host becomes a dependency other teams rely on?

Engineer capacity, isolation, routing, patching, observability, and failure recovery for an inference endpoint you own.

Learning objectives

• Size concurrency and queueing against a stated latency target.
• Isolate the serving workload from the credentials, tenants, and network paths it does not need.
• Define fallback routing and state the failure modes it does not cover.
• Instrument the endpoint so degradation is visible before a user reports it.
• Define who patches the serving runtime, the model artifact, and the host, and how a patch is rolled back.

Study sequence

1. Inference serving as an operated dependency · read · 16 min

   Move from a model that runs to a service others depend on.

2. Compare routing and fallback under failure · compare · 20 min

   Examine each fallback against the failure it must absorb.

3. Rehearse a degradation and a recovery · practice · 35 min

   Rehearse one failure in simulation before production supplies one.

Field exercise

Write the operating plan for one self-hosted endpoint and rehearse the failure you consider most likely. Treat this as a rehearsal even where the chosen ownership level is hosted.

Deliverable: An operating plan for one endpoint plus a written record of one rehearsed failure and its recovery.

Independent challenge: An operations reviewer who did not run the rehearsal reads the plan, states which signal should have revealed the degradation, then checks whether that signal exists.

Evidence to retain

• Capacity and latency targets
• Isolation description
• Routing and fallback rules
• Failure-rehearsal record
• Recovery time and remaining gaps
• Patch and rollback ownership record

05 · Field module

Adapt only with evidence

What measured gap justifies changing the weights instead of changing the context?

Choose among prompting, retrieval, parameter-efficient tuning, full fine-tuning, and pretraining from a reproduced failure rather than an assumption.

Learning objectives

• Reproduce a failure and classify it as missing knowledge, missing format, missing behavior, or missing capability.
• Choose the first intervention from the failure classification and record the measured result before escalating.
• Hold an evaluation set fixed across every intervention so the comparison stays honest.
• State the recurring cost an adapted model adds, including re-evaluation on every base-model change.

Study sequence

1. What adaptation actually changes · read · 15 min

   Separate the failure classes weights can fix from those they cannot.

2. Compare context, retrieval, and tuning on one failure · compare · 22 min

   Match each intervention to the failure class it can repair.

3. Build the evaluation before the training run · practice · 35 min

   Freeze the measurement before any intervention runs.

Field exercise

Take one failing behavior, freeze an evaluation set against it, and record how far the cheapest interventions get before any weight change is considered.

Deliverable: An adaptation decision record holding the frozen evaluation set, the success threshold stated before each intervention ran, the measured result of each escalation step, and the stop condition if the final intervention misses.

Independent challenge: A reviewer who did not run the interventions reruns the frozen evaluation against the learner's best untuned configuration, confirms the reported gap, and checks the escalation record: a weight change proposed without a recorded insufficient result from a cheaper intervention is returned for rework.

Evidence to retain

• Frozen evaluation set
• Failure classification
• Measured result of each escalation step
• Recurring cost and re-evaluation note
• Adaptation recommendation with rejected alternatives

06 · Field module

Govern the model lifecycle

Who owns a model once it is running, and what makes it stop being allowed to run?

Track license, provenance, evaluation, vulnerability, approval, recertification, and retirement for every model a system calls.

Learning objectives

• Record license terms and provenance for each model, including weights obtained through a redistributor.
• Bind every model version to one accountable person rather than a team alias.
• Define the recertification trigger for a model version and the evidence a recertification requires.
• Define retirement, including what happens to workloads still depending on the retired version.

Study sequence

1. Recertification and retirement of a model version · read · 15 min

   Treat an approved model as a dated decision.

2. Compare license and provenance claims across model sources · compare · 20 min

   Compare license, model card, and redistribution path claims.

3. Build a model register with owners and expiry · practice · 35 min

   Make every model in use attributable to a person and a date.

Field exercise

Build a register of every model your systems currently call and give each row an owner, a license record, a provenance record, and an expiry date.

Deliverable: A model register covering every model in use, with owner, exact version, license, provenance, evaluation reference, vulnerability watch source, and expiry.

Independent challenge: A reviewer picks one row and asks the named owner to produce the license text and the provenance record; a row that cannot be evidenced is recorded as unverified.

Evidence to retain

• Model register
• License and provenance records
• Evaluation references
• Recertification schedule
• Retirement plan for one superseded version
• Vulnerability monitoring source per row

Capstone

Private inference decision dossier

Carry one real workload of your own from a traced data path through an ownership decision, a measured hardware and serving plan, an evidence-backed adaptation position, and a governed model register, ending in a named human disposition with a stated reversal condition.

Deliverables

• Data-path map with a store-level access and retention register
• Ownership decision record naming the binding constraint and the rejected alternatives
• Measured hardware fit report and serving operating plan for the chosen level, plus the measurements that ruled out the alternative
• Failure rehearsal record with the observed recovery time and remaining gaps
• Frozen evaluation set and the adaptation position it supports
• Model register with owner, exact version, license, provenance, and expiry
• Named human disposition stating the condition that would reverse it

Verification

• Every measured figure names the hardware, the exact model version, and the date it was taken.
• A reviewer independent of the author reproduces one measurement and one configuration claim from the dossier.
• Unverified license, provenance, and retention entries stay marked as unverified rather than resolved by assumption.
• The privacy boundary is stated as what the configuration shows, not as what the deployment location implies.
• The final disposition is recorded by a named human decision owner rather than by the learner's agent, and states the condition that would reverse it.

Progress and completion are self-directed learning records. They are not certification, professional standing, production evidence, or authorization to deploy an agentic system.

Other published paths

• Engineering with agents without losing engineering
• Agent platform engineering
• Lead AI-native delivery