Start with the problem: why you can't just retrain the whole model per task
Push the problem to the bottom first. You have a 7B open-source LLM and you want it to understand your factory's part-number semantics, read MES fields, and answer maintenance tickets in your format. The obvious move is "train it again on my data" — the term for this is full fine-tuning: treat every weight in the model as trainable, compute gradients on your data, and update all of them.
This road hits a wall, and it hits it at the hardware level. To see why PEFT has to exist, you first have to count exactly how much memory full fine-tuning eats — that bill is the reason the entire PEFT family was invented.
How much memory does one parameter cost under mixed-precision training? Not the naive "one fp16 = 2 bytes":
Per-parameter memory under Adam + mixed precision
┌────────────────────────────┬──────────┐
│ Weight master copy (fp32) │ 4 bytes │
│ Gradient (fp32/fp16) │ 4 bytes │
│ Adam 1st moment m (fp32) │ 4 bytes │
│ Adam 2nd moment v (fp32) │ 4 bytes │
├────────────────────────────┼──────────┤
│ Total │ ~16 bytes│
└────────────────────────────┴──────────┘
7B model × 16 bytes ≈ 112 GB ← activations not even counted yet
65B model × 16 bytes ≈ 1040 GB ← forget doing it on one GPU
The key culprit is that Adam itself stores two states per parameter (moments m and v), each in fp32, on top of an fp32 master weight and gradient — so one parameter balloons to ~16 bytes. A 7B model needs 112GB just for weights + gradients + optimizer states, before the activations saved during the forward pass for the backward pass. An 80GB A100 can't hold a full fine-tune of 7B — that's not bad engineering, that's how expensive this road is mathematically.
So the real question surfaces: do we truly have to touch all 7 billion parameters to teach the model one new task? The entire PEFT (Parameter-Efficient Fine-Tuning) family answers "no"; they only differ on which parameters to move and how.
Fix the levels first: the three-layer frame this article keeps using
Before we dissect anything, let's set up a coordinate system. When fine-tuning any system with a training component, confusing the levels is the most common mistake, so we split it into three layers and tag which layer each method touches:
┌──────────────────┬──────────────────────────────┬────────────────────────────┐
│ Level │ What it is │ Examples │
├──────────────────┼──────────────────────────────┼────────────────────────────┤
│ Mechanism/arch │ the "parts" info flows through│ attention, FFN, lookup, pool│
│ Objective (loss) │ what we "ask" the net to do │ next-token pred, preference │
│ Product │ trained weights & behavior │ updated W, learned prefix │
└──────────────────┴──────────────────────────────┴────────────────────────────┘
One thing to nail down immediately, because it trips up a lot of people: "PEFT" and "what behavior you shape the model into" are two orthogonal axes, not the same thing.
- Axis one (first half of this article): how many parameters move, and where — full FT, Adapter, Prefix/Prompt, LoRA, QLoRA. This axis is only about efficiency; it lives in the mechanism and product layers.
- Axis two (second half): which loss carves the model into which behavior — SFT, RLHF, DPO. This axis is about the objective; it lives in the objective layer.
You can mix freely: LoRA + SFT, QLoRA + DPO, all valid. Many people compare "LoRA" against "RLHF" — that's apples to oranges; they sit on different axes. Hold this split and the rest won't blur.
The theoretical floor: why does "moving only a little" even work?
Before the concrete methods, a more fundamental question: what justifies adjusting only a tiny slice of parameters and getting near-full-fine-tuning results? If that didn't hold in principle, the whole PEFT family would just be a memory-saving compromise. But it has solid theory behind it.
Aghajanyan et al. (2020) proposed and measured a concept called intrinsic dimension[2]. The underlying logic: fine-tuning a pretrained model is essentially finding, in a parameter space of hundreds of millions of dimensions, one point that satisfies the task. They asked — if I don't let you roam the full space freely, but first pick a random d-dimensional subspace and only let you tune within it, how small can d be while still reaching 90% of full fine-tuning's performance?
The result is counterintuitive: on some tasks, RoBERTa needs only about 200 dimensions to reach 90% of full-tune performance[2]. The "effective degrees of freedom" needed to fine-tune are far smaller than the total parameter count. And a second, more important finding: the more thoroughly a model is pretrained and the larger it is, the lower this intrinsic dimension gets — big models aren't harder to tune; they just need a gentle nudge to fall into place.
Full parameter space (hundreds of M dims) Effective intrinsic dim needed
┌───────────────────────┐
│ │ d90 ≈ a few hundred dims
│ ·task solution here │ ⟵ (far below total params)
│ (hiding in a │
│ low-dim subspace) │ bigger model → smaller d90
└───────────────────────┘
This result is PEFT's license. It tells us the weight change ΔW from fine-tuning has "effective information" that is low-dimensional, sparse, and compressible. If the thing we truly need to learn is this low-dimensional, why prepare hundreds of millions of tunable parameters and their optimizer states? Restrict the tunable degrees of freedom to a small structure and you save the vast majority of the cost — which is exactly what every method below does; they just pick different "small structures."
Route one: Adapter — insert a small bottleneck between layers
The first to engineer this idea was the Adapter of Houlsby et al. (2019)[1]. The mechanism is direct: freeze the entire pretrained model, then insert a tiny trainable module inside every Transformer layer, and train only those new modules.
What does the module look like? It's a "bottleneck" structure:
original hidden vector h (dim d = 768)
│
┌─────▼─────┐ down-project: 768 → m (m e.g. 64, far below d)
│ W_down │
└─────┬─────┘
│ nonlinearity (e.g. GeLU)
┌─────▼─────┐ up-project: m → 768
│ W_up │
└─────┬─────┘
│
(＋) ◀── residual skip: add original h back
│
▼ output (still dim 768, size unchanged, plugs back in)
Underlying logic: W_down squeezes 768 dims down to m (e.g. 64), through a nonlinearity, then W_up lifts back to 768, with a residual reconnecting the original input. Only W_down, W_up (plus layernorm) are trainable — about 3.6% of the full model's parameters[1]. Because of the residual, initializing the module near zero means "nothing changed yet," so training starts from a safe point and learns gradually.
Why it works maps straight onto intrinsic dimension: that m-dim bottleneck is the deliberately chosen low-dim subspace, forcing the task's update to squeeze through a narrow channel.
Advantages (derived from the mechanism): each task only stores one small Adapter; one frozen base can host several Adapters for multi-task serving; training doesn't store gradients or optimizer states for the original weights, so memory drops sharply.
Drawbacks (also from the mechanism): an Adapter is serially inserted into the compute path — it deepens the network. Each layer gains three extra steps (down→nonlinearity→up), and they can't be removed afterward. At inference this becomes added latency, especially under model parallelism where these small modules create extra sync points. This "deepens the path, can't be merged away" flaw is exactly the pain point LoRA was built to solve — remember this thread; we'll pick it back up.
Route two: Soft prompts — don't touch weights, change what you feed the model
The second route thinks completely differently: change not a single weight, change the input. But not the text prompt you type — instead, insert into the model's vector space a string of "trainable vectors that correspond to no real word at all." The industry calls this a soft prompt / continuous prompt.
To get it, recall attention's internals. Every Transformer layer computes attention: each token produces query, key, value vectors; the query scores similarity against every token's key to decide which tokens' values to pull information from. The soft-prompt trick is to jam a few trainable vectors at the front of that key/value sequence.
Prefix-Tuning by Li & Liang (2021)[4] is the flagship here. It prepends a trainable prefix to the key and value at every attention layer (implemented as past_key_values):
Normal, KV seen by attention at layer L:
K = [ k1 k2 k3 ... kn ] ← all from real tokens
V = [ v1 v2 v3 ... vn ]
After Prefix-Tuning:
K = [ P^k_1 P^k_2 | k1 k2 ... kn ] ← trainable prefix in front (no real word)
V = [ P^v_1 P^v_2 | v1 v2 ... vn ]
└── only these are trainable; model body fully frozen ──┘
Underlying logic: these prefix vectors are not any real token's embedding — they're pure "steering signals" carved out by gradients. Because they sit in every layer's KV, real tokens' queries attend to them during attention, so at every layer the behavior gets "seasoned" once by this learned signal, nudging the model toward the task. (During training the prefix is often reparameterized through a small MLP for stable convergence, dropped afterward.)
Prompt Tuning by Lester et al. (2021)[5] is the leaner version: prepend the soft prompt only at the bottom input-embedding layer, leaving every middle layer alone. Fewer parameters, simpler. Their key finding is the title — "the power of scale": the bigger the model, the more this simplest soft prompt alone approaches full fine-tuning[5]. This again lines up with intrinsic dimension's "bigger models are easier to tune."
P-Tuning v2 by Liu et al. (2022)[6] then showed: adding the prefix back at every layer (not just input) lets soft prompts robustly approach full tuning across scales and NLU tasks, patching Prompt Tuning's instability on small models.
Advantages: extremely few trainable params (often under 0.1%); one base hosts many prompts; original weights untouched.
Drawbacks (mechanism-dictated): the prefix eats your precious context-length budget — it literally sits in the sequence, consuming usable token space. And optimizing soft prompts is harder and less stable than changing weights, hurting most on small models. Its "capacity" is capped in a few vectors, so the amount of task adjustment it can express has a ceiling.
Route three (the protagonist): LoRA — low-rank decompose the weight change
Now the current industry default. LoRA (Low-Rank Adaptation)[3] turns the theoretical floor above into a minimal mechanism, cleverly dodging Adapter's fatal flaw.
Its starting point is exactly the intrinsic-dimension conclusion: since the weight change ΔW needed to fine-tune has low intrinsic dimension, let's not learn ΔW (a big d×k matrix) directly, but force it to be the product of two skinny matrices.
For an original weight matrix W0 (shape d×k), LoRA freezes it and learns an increment on the side:
W0 frozen (d × k, e.g. 4096 × 4096)
│
input x ───┼────────────────────────┐
│ │
┌─────▼─────┐ ┌──────▼──────┐
│ W0 · x │ │ A: r × k │ ← r tiny, e.g. 8
│ (frozen) │ │ (Gaussian) │
└─────┬─────┘ └──────┬──────┘
│ │ only r dims in the middle!
│ ┌──────▼──────┐
│ │ B: d × r │ ← B initialized to 0
│ └──────┬──────┘
│ │ × (α/r) scaling
└──────────(＋)──────────┘
│
▼
h = W0·x + (α/r)·B·A·x ← ΔW = B·A, rank at most r
Term by term:
W0·xis the frozen original path; not a single gradient flows into it.Ais r×k,Bis d×r, with the middle dimension r (the rank) as small as 4, 8, 16. Their productB·Ayields a d×k matrix, but its rank is at most r — this is "low rank," mathematically forcing ΔW to be a low-dimensional object, exactly matching the intrinsic-dimension insight.Ais Gaussian-initialized,Bis initialized to all zeros. So at the startB·A = 0, ΔW is zero, and the model behaves identically to the original — like Adapter's residual, starting from a safe point.α/ris a scaling coefficient controlling the increment's strength, so tuning r doesn't force you to retune the learning rate.
Only A and B are trainable. For a 4096×4096 matrix with r=8: W0 has 16M parameters, while A+B have only 8×4096 + 4096×8 = 65536 — under 0.4% of the original. The optimizer only stores moments for those ~65k params, and the memory bill is now a different universe.
But LoRA's real crowning point, where it beats Adapter, is a mathematical property at inference:
Training: h = W0·x + (α/r)·B·A·x ← two paths in parallel
│
│ after training, one matrix add
▼
Deployment: W' = W0 + (α/r)·B·A ← fuse into a single matrix
h = W'·x ← identical compute graph to the original!
Because ΔW = (α/r)·B·A has the same shape as W0, you can add them before deployment and merge into one new weight matrix W'. The merged model's compute graph is identical to the original — no extra layer, no extra compute. That is why LoRA's inference latency is zero.
Picking up the earlier thread: Adapter's pain was "serially added module, deepened path, can't be removed, so inference slows." LoRA, by choosing to parallel a low-rank increment of the same shape as the original matrix rather than serialize a new module, can absorb the increment back into the original matrix at deployment, at zero cost. That single mechanistic choice is the root reason LoRA beats Adapter.
LoRA is usually applied only to attention projection matrices (Wq, Wv, sometimes Wk, Wo), because experiments show these take adjustment best.
Advantages (all derived from the mechanism):
- Memory savings are brutal — no gradients or optimizer states for the frozen
W0, only for the sub-1%A,B. - Zero inference latency — mergeable into a single matrix.
- Hot-swappable — one frozen base plus many tiny LoRA files (often just a few MB); swap
A/Bto switch tasks, and you can even serve multiple different LoRAs in one batch.
Drawbacks: low rank is an assumption; when a task is far from the pretraining distribution and genuinely needs large weight rewrites, too small an r caps expressiveness (raise r or switch methods). And picking "which matrices, what r" takes some experience.
QLoRA: crush the frozen base to 4-bit, fine-tune 65B on one GPU
LoRA already solved the "optimizer state" memory. But one chunk remains: the frozen base model itself still has to sit fully in VRAM during the forward pass. A 7B fp16 base is 14GB, 65B is 130GB — LoRA frees you from storing its gradients, but it still has to be "present."
QLoRA by Dettmers et al. (2023)[9] comes to cut this chunk. Core insight: the base is frozen and never updated, so its precision can be crushed low, as long as the forward pass computes accurately enough. QLoRA stacks three techniques:
1. 4-bit NormalFloat (NF4) — a 4-bit datatype tailored for "normally-distributed weights." Its underlying logic is quantile quantization: neural-net weights are empirically near zero-mean normal, so NF4 doesn't cut the range into 16 equal bins but cuts bins by the normal distribution's quantiles, so each bin holds roughly equal numbers of weights. Information-theoretically this is the most efficient split for normal data, preserving the weights' relative relationships in 4 bits (16 levels).
Uniform quantization (naive): NF4 (by normal quantiles):
equal-width bins dense in middle, sparse at tails
├─┼─┼─┼─┼─┼─┼─┤ ├┼┼┼──┼──┼───┼───┤
(tails waste bins) (bins concentrate where weights actually are)
2. Double Quantization — quantization stores a scale (constant) per block of weights. QLoRA quantizes those scales too, saving on average ~0.37 more bits per parameter.
3. Paged Optimizers — borrow NVIDIA unified memory; when a VRAM spike is about to OOM, automatically page optimizer states out to CPU RAM, preventing a crash.
The flow strung together: base weights lie quietly in VRAM as 4-bit NF4; whenever the forward/backward pass needs a chunk of weights, dequantize it to bf16 on the fly to compute, then discard; gradients only flow into the bf16 LoRA A/B. Because the base is never updated, it can stay 4-bit forever. The result is the title's feat — fine-tuning a 65B model on a single 48GB GPU[9], with quality nearly matching 16-bit full fine-tuning.
Note the level relationship: QLoRA doesn't replace LoRA; it compresses LoRA's frozen base. They stack — QLoRA = 4-bit-quantized base + LoRA increment.
Other variants worth knowing: DoRA, (IA)³, BitFit
DoRA (Weight-Decomposed Low-Rank Adaptation)[10] is LoRA's direct upgrade. It first observes that a weight vector can be split into "magnitude" and "direction" — W = m · (V/‖V‖), where m is a scalar magnitude and V/‖V‖ is a unit direction vector. DoRA learns these separately: fine-tune magnitude m directly (very few params), and tune direction with LoRA's low-rank increment. The motivation: analysis shows full fine-tuning's update pattern in magnitude vs. direction differs from LoRA's; by decomposing, DoRA makes the learning pattern hew closer to full tuning, so at the same parameter budget its accuracy is nearer to full tuning. And it keeps LoRA's mergeable, zero-inference-latency benefit.
(IA)³ (Infused Adapter by Inhibiting and Amplifying Inner Activations)[8] goes minimalist: add no matrices, just learn three scaling vectors that element-wise multiply attention's key, value, and the FFN's intermediate activation — amplifying or inhibiting certain dimensions' activity. Fewer params than LoRA, and because it's just element-wise scaling, it too merges into weights with zero inference latency. It comes from the T-Few paper, pitched as "better and cheaper than in-context learning in few-shot"[8].
BitFit (Bias-term Fine-tuning)[7] is the family's most extreme member: train only all the bias terms in the model, freeze everything else. Biases are ~0.08% of parameters, yet on many small tasks they're surprisingly competitive[7]. Its significance is more conceptual than practical — it proves "effective updates are extremely sparse," corroborating the intrinsic-dimension insight from another angle.
Half-time table: cross-comparison of axis one (how many params move)
Putting axis-one methods side by side on the same dimensions (deliberately in a code fence for alignment):
Method Touches which layer Trainable % Inf. latency Mergeable Biggest pain
──────────────────────────────────────────────────────────────────────────────────────
Full FT all weights 100% zero(body) — memory blowup, catastrophic forgetting
Adapter inserted modules ~0.5–4% yes(deepens) no slower inference, unremovable
Prefix/Prompt per-layer/input KV <0.1% yes(eats ctx) no eats context, unstable opt
LoRA parallel low-rank ΔW <1% zero yes low-rank caps big rewrites
QLoRA 4-bit base + LoRA <1% zero yes(dequant) dequant compute overhead
DoRA magnitude+dir low-rk <1% zero yes slightly more complex than LoRA
(IA)³ three scaling vectors <0.1% zero yes smallest capacity
BitFit biases only ~0.08% zero yes(already in) too weak for big rewrites
One line to close axis one: from full FT to BitFit is a spectrum of "using ever-stronger priors to shrink the tunable degrees of freedom ever smaller." Intrinsic-dimension theory guarantees the road is passable, and LoRA — "parallel low-rank, losslessly mergeable" — happens to sit at the sweet spot of efficiency and expressiveness, making it the default.
Switch to axis two: what are you shaping the model into? SFT vs RLHF vs DPO
Every method above answers "how to save." None answers "which data, which loss, to carve the model into which behavior" — that's the objective layer, the orthogonal axis. The same LoRA, paired with different objectives, produces wildly different things.
SFT (Supervised Fine-Tuning) is the most basic objective. Mechanism: prepare a pile of (instruction, ideal response) pairs, and have the model do next-token cross-entropy loss over the ideal-response tokens — the same loss as pretraining, just with your curated demonstration data. It teaches "format and behavior imitation": given this kind of question, answer in this shape. Most PEFT methods above are, in most scenarios, used to do SFT. Its ceiling is clear too: the model can only learn shapes you demonstrated, not relative preferences like "which answer is better" — because cross-entropy only says "this token is right," never "answer A is better than answer B."
RLHF (Reinforcement Learning from Human Feedback) comes to fill that hole; Ouyang et al.'s InstructGPT[11] is the classic recipe, in three steps:
Step1 SFT: first supervised-tune a reasonably obedient initial model on demos.
│
Step2 Train a Reward Model (RM):
have the model generate several answers per question; humans "rank" which is best.
train an RM on these rankings to "score any answer."
│
Step3 Optimize the policy with PPO reinforcement learning:
model(policy) generates → RM scores as reward → PPO pushes policy toward high scores
plus a KL penalty pinning policy near the SFT model (so it won't game the score with nonsense)
Underlying logic: the RM compresses "human preference" into a differentiable scoring function, and PPO (Proximal Policy Optimization) uses that score as a reward signal to push the policy. It's the first time the model can learn "relative better/worse," not just imitate. The cost is a very heavy mechanism: training simultaneously moves the policy, reward model, and reference model (for KL) — several models; PPO itself is hyperparameter-sensitive and unstable; and it runs a "generate→score→update" sampling loop — engineering-complex and compute-expensive.
DPO (Direct Preference Optimization)[12] is the key 2023 simplification, its title the insight: "your language model is secretly a reward model." Its derivation proves that RLHF's "maximize reward + KL penalty" optimum can re-express the reward function via the probability ratio of the policy to the reference model. Substituting it in, the whole RL loop vanishes, leaving a simple classification-like loss that directly consumes (question, better answer, worse answer) preference pairs:
RLHF: data → train RM → PPO sampling loop (policy+RM+reference, 3 models) → aligned
└── heavy, unstable, costly ──┘
DPO: preference data (chosen/rejected) ─→ one closed-form classification loss ─→ aligned
└── no RM, no sampling loop, only policy+reference ──┘
directly raise chosen's relative prob, lower rejected's; reference model is the KL anchor
Underlying logic: DPO no longer explicitly trains a reward model or runs PPO sampling — it does gradient descent directly on preference pairs, raising the log-probability gap of the "better answer" over the "worse answer," while a reference model (usually a frozen copy of the SFT model) serves as the anchor preventing drift. It compresses RLHF's three-model sampling loop into a stable, easy-to-tune, cheap supervised objective via a mathematical equivalence — which is why open-source preference alignment has swung heavily to DPO in the last two years.
Pros/cons (axis two):
Objective Learns what Mech complexity Stability When to use
─────────────────────────────────────────────────────────────────────────────
SFT imitate demo format lowest(one loss) high clear ground-truth/format needs
RLHF relative alignment highest(3 mdl+RL) low, hard ultimate alignment, resources+labelers
DPO relative alignment medium(2 mdl no RL) high default first pick for most alignment
How to choose: a decision map crossing the two axes
Now combine both axes — this is the full decision you meet in practice. Ask axis two first (shape into what), then axis one (how to save):
Q1: which kind is your task?
· just learn a fixed format / domain knowledge / answer per SOP → SFT is enough
· need taste/safety of "this answer is better than that" → preference alignment (SFT then DPO)
· have lots of labelers for fine-grained ranking, want the peak → RLHF (else prefer DPO)
Q2: how much room does your hardware give?
· single consumer GPU (24GB), tune 7B~13B → QLoRA (4-bit base + LoRA)
· have A100/H100, want top quality + fast task switch → LoRA (bf16 base)
· push closer to full-tune accuracy, same param count → DoRA
· very few samples, want the leanest → (IA)³ or Prompt Tuning
· unlimited resources and task very far from pretrain → only then consider Full FT
Grounding a few in your manufacturing scenario:
- Teach the model to read part-number semantics and answer in maintenance-ticket format → this is format and domain imitation; QLoRA + SFT is the best-value opener: one GPU can fine-tune 7B on your ticket corpus, the resulting LoRA is a few MB, and different lines/machine types each store their own LoRA to hot-swap.
- In a RAG pipeline, teach the LLM to "cite the right retrieved chunk, refuse questions it's unsure of" → this is preference and judgment; SFT alone can't teach "refusing beats hallucinating," so use DPO: collect labeler-annotated "good answer vs. hallucinated answer" pairs to align. Do it with LoRA (LoRA-DPO) — cheap and aligned.
- Multiple lines/languages sharing one base model → LoRA/QLoRA's "one base hosting many small adapters" fits exactly: store the base once, one few-MB LoRA per line, keeping deployment and version control clean.
Closing: one logic chain tying the whole article together
From start to finish it's one causal chain:
Full fine-tuning fails because optimizer states inflate each parameter to ~16 bytes — 7B needs hundreds of GB → but Aghajanyan's intrinsic dimension tells us the effective degrees of freedom needed are only a few hundred dims, and lower for bigger models → so "moving only a small structure" holds in principle, and the PEFT family is born → Adapter first realizes it via a "serial bottleneck module," but deepens the path, slows inference, can't be removed → LoRA switches to a "parallel low-rank increment ΔW=BA," losslessly mergeable back to one matrix after training, zero inference latency, solving Adapter's pain in one stroke, becoming the default → QLoRA then crushes the frozen base to 4-bit with NF4, making single-GPU fine-tuning of 65B real → but all of this is only "how to save"; the orthogonal axis "shape into what" is decided by SFT (imitation), RLHF (preference but heavy mechanism), DPO (a mathematical equivalence compressing RLHF into a stable supervised preference loss) → in practice you cross the two axes: most scenarios = QLoRA/LoRA to save resources × SFT to bootstrap, DPO to align.
One line for the whole article: PEFT answers "how much moves," SFT/RLHF/DPO answer "shape into what"; LoRA reigns because it turns the intrinsic-dimension insight into a low-rank parallel increment that's cheap to train, free at inference, and hot-swappable.
Extension hooks
To go deeper, pick from: how to sweep LoRA's rank r and α and how much "which matrices" matters; QLoRA dequantization's real inference-time compute cost and how engines like vLLM handle it; DPO's variants (IPO, KTO, ORPO) and which assumptions each changes; and in RAG, the trade-off of "fine-tune the LLM vs. spend the effort on retrieval and prompting." Any one could be its own article.
References
[1] Houlsby, N. et al. (2019). Parameter-Efficient Transfer Learning for NLP (Adapter). — https://arxiv.org/abs/1902.00751
[2] Aghajanyan, A. et al. (2020). Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. — https://arxiv.org/abs/2012.13255
[3] Hu, E. J. et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. — https://arxiv.org/abs/2106.09685
[4] Li, X. L., Liang, P. (2021). Prefix-Tuning: Optimizing Continuous Prompts for Generation. — https://arxiv.org/abs/2101.00190
[5] Lester, B., Al-Rfou, R., Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. — https://arxiv.org/abs/2104.08691
[6] Liu, X. et al. (2022). P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks. — https://arxiv.org/abs/2110.07602
[7] Ben-Zaken, E., Ravfogel, S., Goldberg, Y. (2021). BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models. — https://arxiv.org/abs/2106.10199
[8] Liu, H. et al. (2022). Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning ((IA)³ / T-Few). — https://arxiv.org/abs/2205.05638
[9] Dettmers, T. et al. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. — https://arxiv.org/abs/2305.14314
[10] Liu, S.-Y. et al. (2024). DoRA: Weight-Decomposed Low-Rank Adaptation. — https://arxiv.org/abs/2402.09353
[11] Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback (InstructGPT / RLHF). — https://arxiv.org/abs/2203.02155
[12] Rafailov, R. et al. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. — https://arxiv.org/abs/2305.18290