thinkingmachines
Inkling-Small
image text to textapache-2.01.2M downloadsHugging Face
No server in stock right now has the 594 GB this model needs for inference.
GPU memory it needs (estimate)
Rule of thumb from the model's size. Real needs depend on batch size, context length and framework.
Inference (full precision)
594 GB
Inference (8-bit)
297 GB
Inference (4-bit)
149 GB
LoRA fine-tuning
644 GB
QLoRA (8-bit base)
322 GB
Full fine-tuning
2378 GB