tencent
Hy4-preview
text generationapache-2.011.5K downloadsHugging Face
No server in stock right now has the 1743 GB this model needs for inference.
GPU memory it needs (estimate)
Rule of thumb from the model's size. Real needs depend on batch size, context length and framework.
Inference (full precision)
1743 GB
Inference (8-bit)
872 GB
Inference (4-bit)
436 GB
LoRA fine-tuning
1889 GB
QLoRA (8-bit base)
944 GB
Full fine-tuning
6973 GB