2026-11-11 · America/Los_Angeles · 社区动态 · #17

I challenged llama.cpp on a 4GB RTX 3050 laptop with my own Rust inference engine. Here's what happened....

Hi all. For the past few weeks I’ve been building ge, a GGUF inference engine with its own CPU kernels (int8, AVX2), its own Vulkan GPU kernels, and an automatic CPU/GPU split for models that don’t fit in VRAM. It’s a single ~4 MB executable with an OpenAI and Anthropic-compatible server built in. I tried to make the comparison fair: same GGUF files, back-to-back runs, and llama.cpp’s own llama-bench . Machine: Ryzen 5 5600H, 16 GB DDR4-3200, RTX 3050 Laptop 4 GB at a 35 W power limit (low end for this GPU), Windows 11, llama.cpp b9102 (Vulkan). Where ge wins CPU prompt processing (Qwen3-4B…

热度 44.3 / 100；排名与评分保留该期记录。