2026-11-07 · America/Los_Angeles · 社区动态 · #12

I ran a 63-case eval on my English/Arabic LLM service. Same model, same harmful topics, different decisions depending on the language

I built a small English/Arabic quote service using Gemini and created 63 evaluation cases to test its guardrails. A few findings from testing with gemini-3.5-flash-lite: Three unexpected bugs: A model refusal was displayed as a quote, a Chinese word slipped into an Arabic response, and stalled API calls failed requests despite having half their time budget remaining. Prompt injection: The input checks blocked 15 out of 20 attacks. Gemini ignored the remaining five, as well as all 17 canary attacks tested without guardrails. This suggests the checks provide an additional layer of protection…

热度 44.6 / 100；排名与评分保留该期记录。