tencent
Hy3
text generationapache-2.01.2M downloadsHugging Face
No server in stock right now has the 668 GB this model needs for inference.
GPU memory it needs (estimate)
Rule of thumb from the model's size. Real needs depend on batch size, context length and framework.
Inference (full precision)
668 GB
Inference (8-bit)
334 GB
Inference (4-bit)
167 GB
LoRA fine-tuning
723 GB
QLoRA (8-bit base)
362 GB
Full fine-tuning
2671 GB