2026-11-16 · America/Los_Angeles · 论文 · #18
Interpreting Coding Agents Through Their Behavioral Trajectories
Standard coding-agent benchmarks primarily evaluate binary task success, treating execution as a black box and obscuring the behaviors that produce those outcomes. We analyze a large corpus of 143,209 execution trajectories from 125 agent configurations spanning 117 unique models. To enable comparisons across heterogeneous agent interfaces, we develop a rule-based parser that maps raw execution traces to high-level actions such as repository search, file editing, and test execution, and audit its predictions against a six-model consensus reference on 500 events. We find that agents with…
热度 39.8 / 100;排名与评分保留该期记录。