2026-10-19 · America/Los_Angeles · 社区动态 · #9

AI labs report benchmark scores 6.4% higher than independent testers, on average

I've been building whatmodel.sh, an open source AI leaderboard, and wanted to see how a labs' own benchmark scores compare with independent testers. Using 1,629 published benchmark results, I matched 611 comparisons of 180 models on 103 benchmarks. AI labs reported scores 6.4% higher on average for the same models on the same tests. Everything is open: the dataset is CC BY-SA 4.0 and the code is MIT. Sources and their licenses are recorded, with links on each model's page. Interactive chart: https://whatmodel.sh/claims Data and code: https://github.com/jakemanger/whatmodel Please star on…

热度 46.0 / 100；排名与评分保留该期记录。