2026-11-04 · America/Los_Angeles · 社区动态 · #11

I removed 85% of gemma's weights (and it is mostly fine)

I pruned Gemma 4 26B-A4B, an MoE model, down to 15% of its original size, and the model still does great on one narrow task. It gets 92% agreement with the original on a classification task. That’s close to how often the 12B agrees with the 26B on the same ticks, which is 90%. This isn’t a model for general tasks, and tasks that require knowledge suffer because the knowledge is just gone. It was made for my app that runs an on-device LLM 24/7, so that makes memory footprint and energy consumption extremely important. On my Mac it hits 79 tokens/s, which is roughly double the speed of the…

热度 46.8 / 100；排名与评分保留该期记录。