2026-10-20 · America/Los_Angeles · 社区动态 · #14

Improve Qwen3.8-27b low quants by using a prefiller model that streams from SSD

This demo from ISTA-DASLab recovers low quant accuracy while retaining low VRAM usage and fast decoding. It needs a Nvidia Blackwell GPU to run, so I have not tested it. For context longer than ~4k there should be no speed penalty compare to running without the prefiller model. Graph shows MMLU-Pro accuracy. Huggingface🤗: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-NVFP4-prefiller

热度 46.2 / 100；排名与评分保留该期记录。