2026-11-05 · America/Los_Angeles · 论文 · #10

SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding

Speculative decoding (SD) accelerates large language model inference by reducing the sequential cost of token-by-token generation. Advanced SD methods reduce proposal cost with parallel drafters that predict entire blocks in one forward pass, but overall speedup still depends on acceptance length. On-policy distillation (OPD) can push well-trained drafters beyond acceptance-length plateaus, yet conventional pipelines collect supervision through sequential draft-and-verify rollouts. Our key insight is that on-policy supervision need not inherit these dependencies: a draft block can be…

热度 41.3 / 100；排名与评分保留该期记录。