2026-10-18 · America/Los_Angeles · 论文 · #2

Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design

Looped language models reuse physical Transformer blocks across logical depth, but established architectures entangle recurrence placement, input injection, and recurrent state organization. We isolate these axes in models trained from scratch at matched parameter scale, logical depth, and token budgets, and then test whether the resulting design principles transfer to resident- and active-parameter-matched Mixture-of-Experts (MoE) models. The prelude-recurrent block-coda sandwich architecture consistently improves performance on context extraction and reasoning-intensive tasks. Input…

热度 20.3 / 100；排名与评分保留该期记录。