2026-11-08 · America/Los_Angeles · 社区动态 · #3

Run MoE models bigger than your RAM on phone, laptop or GPU: it places every weight on VRAM, RAM or flash by itself for max prefill and decode speed (built on llama.cpp)

Hi all, in the last months I worked on [BigMoeOnEdge](https://github.com/Helldez/BigMoeOnEdge), it's built on top of llama.cpp. It runs MoE models that are bigger then the memory of the device, and it configure itself for the machine where it runs. It works with any MoE GGUF that llama.cpp can load, any quantization, no code per model. OS: Android, iOS, macOS, Windows, Linux (arm64 and x86_64). Backends: CPU, CUDA, ROCm (HIP), Vulkan, Metal, OpenCL, Hexagon NPU. Other llama.cpp backends load in the same way if you build them. The first image: a 16 GB Apple silicon laptop, Qwen3.6-35B-A3B…

阅读原文查看评分、趋势与关联来源阅读整期日报

Run MoE models bigger than your RAM on phone, laptop or GPU: it places every weight on VRAM, RAM or flash by itself for max prefill and decode speed (built on llama.cpp)

热度 49.7 / 100；排名与评分保留该期记录。