2026-11-10 · America/Los_Angeles · 官方发布 · #15

Announcing Long-Context Training on TPUs with MaxText Context Parallelism

Google Cloud's MaxText framework now supports long-context training up to 14 million tokens on Cloud TPUs by implementing four context-parallel strategies: All-gather, A2A, P2P, and USP. Powered by the Tokamax Splash Attention kernel, these strategies effectively manage attention compute and memory bottlenecks without materializing full key-value sequences in high-bandwidth memory. Developers can adopt these capabilities in MaxText v0.2.4 to achieve high Model FLOPs Utilization (MFU) across large-scale model training workflows.

热度 39.0 / 100；排名与评分保留该期记录。