AI Resonance · 阅读最新日报 · AI 入门推荐

2026-11-16 · America/Los_Angeles · 社区动态 · #14

I made a small library for comparing agent versions (success vs cost plots, failure maps). What do you use?

Repo: https://github.com/slavadubrov/harness-evals It's for one question: I changed my agent (a prompt, a tool, an approval step), did it get better? You write tasks with checks and a small adapter that runs your agent. It runs each version several times and gives you success, pass^k, cost and latency with intervals, a success-vs-cost plot, and a failure map that shows which task and check each version fails. No dependencies, MIT. Install from github: pip install "git+https://github.com/slavadubrov/harness-evals@v0.5.1" I also wrote an article that walks through how I use it:…

I made a small library for comparing agent versions (success vs cost plots, failure maps). What do you use?

热度 38.4 / 100;排名与评分保留该期记录。