2026-11-04 · America/Los_Angeles · 社区动态 · #10

Can a 1-trillion-parameter model run on a 64 GB desktop? I built an open-source engine to find out, and it already beats llama.cpp on models bigger than RAM.

Kimi K2 has 1 trillion parameters but uses only about 32 billion per word. Banach keeps the right experts in RAM and streams the rest from NVMe, so the model doesn't have to fit in memory. Measured: a 22 GB model on 16 GB of RAM. llama.cpp runs out of memory; Banach is 2.2x faster at prompt processing, with byte-identical output. Trillion-parameter estimate: banach plan says Kimi K2 would run at about 3.6 tok/s on a 64 GB desktop with an RTX 5080. That's an estimate, not a measured run. pip install banach https://github.com/gautam1858/banach Got a 64 GB+ machine? Run Kimi K2 and post your…

阅读原文查看评分、趋势与关联来源阅读整期日报

Can a 1-trillion-parameter model run on a 64 GB desktop? I built an open-source engine to find out, and it already beats llama.cpp on models bigger than RAM.

热度 45.0 / 100；排名与评分保留该期记录。