2026-11-09 · America/Los_Angeles · 社区动态 · #17

Built a lightweight, open-source RAG evaluation tool — looking for developer feedback

I've been working on a small Python-based RAG evaluation utility for developers who want to test their pipelines against a golden dataset without setting up a full evaluation platform. The idea is deliberately simple: run evaluations locally, inspect retrieval and answer-quality metrics, and identify where the pipeline is failing. It currently includes: Retrieval Recall@K Answer faithfulness and relevance metrics Exact match and token F1 Latency and cost reporting Optional LLM-as-judge evaluation The core metrics don't require an API key. One issue I encountered was how unanswerable…

热度 37.7 / 100；排名与评分保留该期记录。