2026-11-15 · America/Los_Angeles · 社区动态 · #4
Basalt 0.4.0: More models, still as fast - Qwen 27B up to 770 tok/s on a 5090
Hi everyone! I wanted to post a quick update on Basalt . Since last I posted, multi model and router support was implemented, starting with DeepSeek V4.1 Flash (IQ1_M 90 tok/s conv, 246 tok/s struct on a 5090 + 5060 Ti). Also new: video support, files in chats, and a bunch of fixes and improvements. Now 0.4.0 is out featuring Basalt's first dense model, Qwen 3.8 27B NVFP4. It's fast and accurate, top-1 92.8% vs the full BF16 model. This, in preparation for Qwen 4, so we had a headstart on the dense kernels. The numbers: Up to 776 tok/s on structured output, 262 conversation, 194 on prose.…
热度 46.3 / 100;排名与评分保留该期记录。