2026-10-22 · America/Los_Angeles · 社区动态 · #7

capy-flash v0.7.1: Qwen3.8-Flash-Next (125B MoE) now runs in 64 GB of RAM (234 tok/s on two 16 GB cards), and Qwen3.8-27B 3-bit fits on a single 16 GB card (174 tok/s)

I've been building capy-flash, an open-source inference engine for running Qwen3.8-Flash-Next (125B MoE) and Qwen3.8-27B on consumer RTX 50 cards. Until now it needed my setup (two 16 GB cards and 96 GB of RAM) or better. v0.7 is about smaller machines: start with what you have, add a second card later, same engine and same model files. What's new Flash-Next on 64 GB of RAM, one or two GPUs. The routed experts are ISTA-DASLab's GSQ-RCO i-quants (~3 bit, copied as they are, not re-quantized): 48 GiB in RAM instead of 65 for NVFP4. The hot experts sit in VRAM, the rest are computed on the CPU.…

热度 47.6 / 100；排名与评分保留该期记录。