meta-llama

Llama 3.1 8B Instruct

Meta's widely deployed 8B instruction-tuned model, often used as a small general-purpose baseline.text generationllama3.1Access request needed4.5M downloadsHugging Face

Cheapest GPU in stock that fits

A single GPU with at least 16 GB for inference.

NosanaIn stock · 9 freeStart right away

1× RTX 309024 GB VRAM

Nosana network, hosts worldwide

Details

Stock and price checked 17 min ago.

Self-service: you start it yourself, with no order or sales contact.

nvidia-3090

Full price

€0.17

per hour, whole instance, 1 GPU

View offer

License restrictions

Services above 700M monthly active users need a separate license from Meta. Repo is gated.

GPU memory it needs

From model cards and vendor documentation. Lower precision needs less memory, with slightly lower quality. Estimates are marked.

What you want to do	Precision	Minimum	Basis
Run the model Hugging Face figure for loading weights, excluding framework overhead; the KV cache adds about 2 GB at 16k tokens.	FP16, full	16 GB	Published
Run the model vLLM recipe minimum for this repo's BF16 checkpoint.	BF16, full	20 GB	Published
Run the model vLLM recipe minimum for separate repo RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8.	8-bit	10 GB	Published
Run the model vLLM recipe minimum for separate repo hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4.	4-bit	5 GB	Published
LoRA fine-tuning Hugging Face estimate for LoRA fine-tuning, with no batch size given.	FP16, full	16 GB	Published
LoRA fine-tuning Hugging Face estimate for QLoRA on a 4-bit base, with no batch size given.	4-bit	6 GB	Published
Full fine-tuning Hugging Face estimate for full fine-tuning, with no batch size given.	FP16, full	60 GB	Published