thinkingmachines
Inkling
image text to textapache-2.01.7M downloadsHugging Face
No server in stock right now has the 2129 GB this model needs for inference.
GPU memory it needs (estimate)
Rule of thumb from the model's size. Real needs depend on batch size, context length and framework.
Inference (full precision)
2129 GB
Inference (8-bit)
1064 GB
Inference (4-bit)
532 GB
LoRA fine-tuning
2306 GB
QLoRA (8-bit base)
1153 GB
Full fine-tuning
8515 GB