2026-11-11 · America/Los_Angeles · 社区动态 · #3

Evaluating small LLMs on Android with Jupyter and Pydantic Evals

I've been working on PythonHere, a project that lets me run Python code on Android directly from Jupyter. Recently I added LiteRT-LM support, as I wanted to explore what small models can actually do on a phone. For benchmarking, I tried Pydantic Evals. It works with ordinary Python functions, so I could run inference on the phone and write my own evaluators in Jupyter. I found this combination quite convenient for experimenting with models and checking their results. I tried Qwen2.5-Coder-3B-Instruct on a few Python tasks. I wasn't expecting much from a 3B model, but it followed instructions…

热度 40.3 / 100；排名与评分保留该期记录。