2026-11-03 · America/Los_Angeles · 社区动态 · #2

Defer, a Deterministic LLM Inference Engine

I created a an experimental deterministic LLM inference engine called Defer . When given the same prompt, model and parameters such as temperature and seed the engine will always return the same result. Its slow and single-threaded, about 17 tokens/s for SmolLM2-135M on my laptop, but every token, logit and KV-cache entry matches bit for bit every time you run it. I've tried making LLMs deterministic before but was never successful. Even after adjusting the temperature and using a fixed seed I would still get different results from run to run. Defer gets around this by running every…

热度 44.6 / 100；排名与评分保留该期记录。