# Parameter-efficient fine-tuning A family of adaptation methods that trains a relatively small subset of parameters or added components while leaving most or all pretrained model weights fixed. technical-term · Models & inference · foundational · Reviewed 2026-08-09 ## Definition Parameter-efficient fine-tuning, or PEFT, adapts a pretrained model without updating every base parameter. Methods include trainable prompts, selective layer tuning, and adapters such as Low-Rank Adaptation, which adds trainable low-rank matrices while freezing base weights. PEFT can reduce training memory and artifact size, but it does not guarantee full-fine-tuning quality, eliminate data risk, or remove the need for evaluation. PEFT and LoRA are established techniques with broad tooling support; results and operational tradeoffs vary by architecture, method, data, and task. ## Why it matters Keeping the base model fixed makes it possible to maintain smaller task-specific adapters and sometimes combine or switch them at serving time. The deployed behavior depends on the exact base model, adapter, tokenizer, runtime, and composition order, so all of them belong in artifact lineage. ## System anatomy - **Frozen base:** Most pretrained model parameters remain unchanged during adaptation. - **Trainable component:** A smaller parameter subset, prompt, layer selection, or adapter learns the target change. - **Adapter artifact:** The compact learned state tied to a compatible base model and configuration. - **Composition:** Rules for loading, merging, switching, or combining adapters during deployment. ## Important distinctions - **Full fine-tuning:** Full fine-tuning updates all or most base parameters; PEFT intentionally limits the trainable portion. - **Quantization:** PEFT changes learned behavior; quantization changes numerical representation primarily for storage or execution efficiency. ## Implementation signals - Benchmark PEFT against prompting, retrieval, and full fine-tuning baselines - Version the adapter with its exact base model and tokenizer - Evaluate merged and unmerged deployment forms separately ## Failure modes - An adapter is loaded onto an incompatible base revision - Lower training cost is mistaken for lower assurance burden - Multiple adapters interact in untested ways ## Related knowledge - [Model adaptation](https://aisdlc.ai/agentic-engineering/model-adaptation) — The deliberate modification of a model or its task-facing behavior for a target domain, task, language, policy, or operating environment using measured evidence. - [Quantization](https://aisdlc.ai/agentic-engineering/quantization) — The representation or computation of model values at lower numerical precision to reduce memory, storage, bandwidth, or inference cost, with workload-dependent tradeoffs. - [Open-weight model](https://aisdlc.ai/agentic-engineering/open-weight-model) — A model whose trained parameter weights are available for download and use under stated license terms, enabling inspection, adaptation, or deployment outside a hosted API. - [Inference serving](https://aisdlc.ai/agentic-engineering/inference-serving) — The runtime and operational layer that loads model artifacts, accepts requests, schedules computation, executes generation, and returns outputs under defined service constraints. ## Sources and further study - [Hugging Face — Parameter efficient fine-tuning methods](https://huggingface.co/docs/peft/main/methods/overview) — Catalogs parameter-efficient adaptation methods, including soft prompting, selective layer tuning, adapters, and Low-Rank Adaptation variants supported by the PEFT library. **Use:** Official guidance. **Limitation:** Library documentation describes available methods, not equal quality across tasks or parity with full fine-tuning; the main-version page may also change ahead of stable releases. - [arXiv / ICLR — LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685) — Introduces Low-Rank Adaptation, which freezes pretrained weights and injects trainable low-rank matrices to reduce the number of parameters updated for downstream adaptation. **Use:** Primary definition. **Limitation:** The paper reports selected models, tasks, and hardware conditions; it does not establish that LoRA matches full fine-tuning or preserves safety for every workload. - [Hugging Face — Fine-tuning](https://huggingface.co/docs/transformers/en/training) — Defines fine-tuning as continued training of a pretrained model on a smaller task- or domain-specific dataset and provides an implementation path through Transformers. **Use:** Official guidance. **Limitation:** The tutorial does not settle data rights, evaluation design, safety, governance, or deployment fitness; adaptation can introduce regressions and requires use-case testing. --- This library synthesizes cited research, standards, official documentation, and clearly attributed practitioner perspectives. Maturity describes the state of a concept—not vendor endorsement, production readiness, or permission to deploy. Benchmarks and demonstrations do not replace use-case evaluation, governed controls, independent verification, or named human release authority.