2026-11-09 · America/Los_Angeles · 社区动态 · #12

When is it time to move from a custom LLM proxy to a proper gateway?

I initially built a small FastAPI proxy for one of our apps so we could switch between OpenAI and Anthropic without having to change anything in the application code. It worked fine when we only had one app using it. Now we're at four apps across three teams, all sharing the same two provider API keys, and the setup is getting harder to maintain. We're starting to run into issues around team-level API keys, usage limits, tracking requests properly, and keeping sensitive customer data out of prompts before they reach external providers. Right now, most of our logging is handled through a…

热度 44.3 / 100；排名与评分保留该期记录。