2026-10-25 · America/Los_Angeles · 社区动态 · #12

DFlash2 on Qwen3.8-27B: 154 tok/s on code, ~35 on prose. How do you fix prose acceptance?

Been pushing Qwen3.8-27B on Kaggle's free 2x T4 (no P2P, 70W cap per card) for a while. Current setup: EXL3 4.0bpw on exllamav3, tensor parallel over PCIe, DFlash2 block drafter (7 tokens), greedy and exact vs no-draft. Numbers over a 38 prompt suite: - overall 55-57 tok/s (no draft is ~18-19) - HumanEval 154 tok/s - prose 35-37 tok/s For comparison on the same 2 T4s: llama.cpp best was 16.5 (MTP crashed on tensor split), vLLM int4 + its own MTP drafter did 40.8. Prose is the wall. Acceptance sits around 2.5 tokens per step, vs ~7 on code. Questions: 1. Has anyone got prose acceptance above…

热度 45.6 / 100；排名与评分保留该期记录。