2026-11-07 · America/Los_Angeles · 社区动态 · #11

MaxxBench (RLVR'd LLMs fail easy variants of DeepSWE tasks + obsess over "hidden tests" and pleasing the grader)

Similar to Kimi K3.1 Pro Recalls and Enforces Hidden Test Rules Some of these drops are very big: DeepSeek V4 Flash falls from 53.3% to 28.1% on a lightly different version of Deep-SWE. (how different? "one DeepSWE task asks an agent to write a separate test report for each browser. We added a requirement that the filenames be lowercase. The reporting feature still needs to work; Chrome.xml should now be chrome.xml. Another task adds retirement dates to FastAPI routes. We changed the required timestamps to UTC with a trailing Z, instead of preserving the supplied time-zone offset.") Full of…

热度 45.0 / 100；排名与评分保留该期记录。