2026-10-14 · America/Los_Angeles · 社区动态 · #16

Tracing where LLM "slop" style comes from in the model and in training, and cutting it out [R]

I wanted to find where the recognizable assistant style (em dashes, "it's not X, it's Y", lists of three, buzzwords) lives in a model and which training stage installs it. Where it lives: in Qwen3-4B-Instruct, a difference-in-means direction from the model's own exaggerated vs plain writing separates 64 hand-written minimal pairs. It correlates up to 0.82 with a trope-based slop score on normal output. Logit-lens readouts show a register axis (grand, abstract vocabulary vs plain hedges like "really", "basically", "stuff"), and per-trope directions are cross-lingual: "not X, but Y" promotes…

热度 47.5 / 100；排名与评分保留该期记录。