AI Resonance · 阅读最新日报 · AI 入门推荐

2026-11-18 · America/Los_Angeles · 论文 · #5

Multi-Branch Transformer

Transformer language models are commonly scaled by making their blocks wider or stacking them deeper. In this work, we explore a complementary path: branching, the parallel composition of full Transformer blocks. We introduce the Multi-Branch Transformer (MBT), which processes the input through shared initial layers, executes independently parameterized branches of sequential blocks in parallel, and fuses their outputs before shared final layers. At comparable training compute, we study three MBT variants: Wide-Shallow preserves block width, Narrow-Deep preserves sequential depth, and…

Multi-Branch Transformer

热度 47.9 / 100;排名与评分保留该期记录。