2026-11-04 · America/Los_Angeles · 社区动态 · #9

I cut my LLM-as-judge eval costs ~70% with prompt-cache engineering and a cheap decision-model cascade

I have put quite a lot of thought on running evals, both in terms of their correctness, but also on how to make them cheaper. And this post is on the latter, what I have learned running thousands of LLM-as-judge checks, and what actually moved the bill. If running only one test, the options are limited, but when running a lot of similar tests, we get a lot of good engineering practices to prompt structures, cache point selections, and even the System One models to get the price for individual checks as low as possible. (I'm also building an eval tool called Apo, but all of this works with…

热度 45.5 / 100；排名与评分保留该期记录。