2026-11-05 · America/Los_Angeles · 社区动态 · #12

Retrieval finds the fact. Who tracks the change? Ai2's StateBench and what it suggests for how we store agent state

Ai2's StateBench (Nov 3) separates two things we often lump together under "long context": finding a value, and following how values change. The setup is synthetic: the model reads a long sequence of variable assignments, then reports the final value of every variable. Frontier models (Gemini 3.8 Flash, Claude Opus 5.5, GPT-6 Sol and Luna) did well when the task was "latest assignment wins". When updates built on each other, like running totals or one variable taking another's current value, accuracy fell sharply as inputs got longer, to 13% or lower on the longest hardest problems. The…

热度 38.7 / 100；排名与评分保留该期记录。