Models / Run locally
Run Kimi K3 locally
Moonshot AI · ≈1T-class MoE (≈32B active) · open weights
A trillion-parameter class model: ≈580GB at 4-bit. This is rack hardware — locally it's for labs; everyone else calls it metered and pays cents.
By quantization
Hardware requirements
| Quantization | ≈ File size | ≈ Memory needed | Runs on |
|---|---|---|---|
| Q4_K_M (4-bit) | 580 GB | 667 GB | Multi-node GPU clusters — datacenter serving only |
| Q8_0 (8-bit) | 1070 GB | 1231 GB | Multi-node GPU clusters — datacenter serving only |
| FP16 (full) | 2100 GB | 2415 GB | Multi-node GPU clusters — datacenter serving only |
Sizes are computed from the ≈1000B parameter count with standard GGUF math and ~15% runtime overhead — verify against the model card before buying hardware. As a mixture-of-experts model only ≈32B parameters are active per token, which helps speed — but the full weights still have to fit in memory.
Local setup
How to run it
Grab a community GGUF conversion from Hugging Face and serve it with llama.cpp, LM Studio, or (for multi-GPU rigs) vLLM with the original weights. Pick the quantization from the table above that fits your memory.
For serious throughput on server hardware, vLLM with tensor parallelism is the standard serving stack; llama.cpp and MLX (Apple silicon) win below that line.
The API route
Or skip the GPU entirely
The same model is one API call away at $1.2 / M input · $3.6 / M output — no download, no VRAM math, metered to the exact call. Free starter credits cover your first runs, and because the weights are open, nothing locks you in: start metered, move to your own hardware if sustained volume ever justifies it.
FAQ
Frequently asked questions
Can my GPU run Kimi K3?
Kimi K3 is ≈1T-class MoE (≈32B active). At 4-bit quantization you need roughly 667GB of memory (multi-node gpu clusters — datacenter serving only). A trillion-parameter class model: ≈580GB at 4-bit. This is rack hardware — locally it's for labs; everyone else calls it metered and pays cents.
Is there a Kimi K3 GGUF?
Open-weight releases in this family get community GGUF conversions on Hugging Face shortly after release — search the model name plus "GGUF" and pick the quantization your memory allows from the table above.
Is running Kimi K3 locally cheaper than the API?
Only at sustained volume. The API price is $1.2 / M input · $3.6 / M output with no hardware, electricity, or ops cost — a machine that can serve this model costs more per month idle than most teams' entire inference bill. Prototype metered, self-host when utilization justifies it.
Can I use the outputs commercially?
Commercial output use permitted per Moonshot terms.
Explore
Run other models locally
One key. Every model. Exact prices.
Prototype on Kimi K3 with free starter credits while the weights download. Free starter credits included — no subscription required.