What does it cost to fine-tune a 7B model?
For a 10M-token dataset with LoRA on a single A100, that is about 1.85 GPU-hours, roughly $1.57 at median prices, and a full fine-tune on eight H100s is only $18. QLoRA on a single 4090 is not cheaper ($2.12, 36% above LoRA) — the slowdown from quantisation eats the cheaper card, so pick it for 24GB of VRAM, not to save money. The compute is trivial; what costs money is the twentieth attempt.
Prices: median device-level listings for A100 SXM4 / H100 SXM / RTX 4090 · as of 2026-11-05 · throughput and VRAM needs are published-baseline assumptions
Three recipes, three bills
| 做法 | 显存 | 吞吐假设 | 时长 | 卡时 | 成本 | 说明 |
|---|---|---|---|---|---|---|
| QLoRA · 1x RTX 4090 24G | 24 GB | 600 tok/s | 4.63 h | 4.6 | $2.12 | 4-bit quantised, lowest VRAM |
| LoRA · 1x A100 80G | 80 GB | 1,500 tok/s | 1.85 h | 1.9 | $1.57 | common setup |
| Full fine-tune · 8x H100 | 8 × 80 GB | 3,000 tok/s | 0.93 h | 7.4 | $17.69 | conservative throughput |
统一按 1,000 万 token 训练集、单轮训练计；卡时 = 时长 × 卡数。吞吐取保守值，实际用 FlashAttention / packing 通常更快，账单会更低。
Iterations are the real cost
| 实验轮数 | LoRA 算力成本 | 累计占比说明 |
|---|---|---|
| 1 轮 | $1.57 | 一次跑通 |
| 20 轮 | $31 | 调学习率 / 数据配比 / epoch 的常见量级 |
| 50 轮 | $78 | 要追指标时的量级 |
把这张表和第一张对比：50 轮 LoRA 的钱，仍然低于一次全参微调乘 3 轮。这就是实践里的正确顺序——先用 LoRA 把数据与评测打磨到收敛，再决定要不要上全参。
How it is computed
卡时 = 训练 token 数 ÷ 每秒 token 数 ÷ 3600 × 卡数；成本 = 卡时 × 单卡时价
显存需求按「权重 + 优化器状态 + 激活」估算：7B 的 bf16 权重约 14GB，全参训练要存梯度与 Adam 状态，单卡需要 80GB 以上；LoRA 只训练低秩旁路，单张 80G 足够；QLoRA 4-bit 量化后 24G 就能跑。
Caveats
② 不含评测与部署。推理服务要另算（见「70B 推理成本」那一页的口径）。
③ 长上下文会让成本非线性上升。序列长度翻倍，注意力开销按平方增长，本页按常规 2–4k 上下文估算。