How much VRAM do you need to run or fine-tune a 70B model?
A 70B model needs 40 to 141 GB for weights plus a KV cache to serve, and 50 GB to 1.1 TB to fine-tune. The math, GPU setups and a script to check yours.
fine-tuning
2 posts with the tag “fine-tuning”
Fine-tuning on Nodus: 67% lower charges
Nodus and Modal reached 91.74% accuracy with identical trained weights. We compare successful-run charges, retry costs and delivery time.