2026-11-01 · America/Los_Angeles · 论文 · #20

Sparse Text Diffusion under Dual Budgets: Context Connectivity and Expert Activation

Long-context diffusion repeatedly pays for both token interaction and feed-forward computation. We formulate these costs as two separate budgets: the edges available to an attention graph and the experts activated for each token. This view connects local, dilated, and global attention with conditional expert computation without assuming that sparsity alone preserves document-level information. We specify normalized masked attention, distinguish denoising depth from sampling iterations, and derive graph and compute bounds. A reported sparse-expert configuration, denoted DualBudget, is…

热度 12.6 / 100；排名与评分保留该期记录。