2026-10-24 · America/Los_Angeles · 社区动态 · #13

High-Performance llama.cpp fork for Legacy GPUs: Kepler Support, CUDA Graphs & Windows NCCL!

I've been working on improving llama.cpp support and performance on older NVIDIA GPUs, particularly Kepler and other pre-Ampere architectures. The goal is simple: get more life out of older CUDA hardware instead of treating it as e-waste. Achieved Performance On MTP Qwen3.6-35B-A3B with Q8_0 quantization, I've observed the following (generation) speeds using 4x Tesla K40c GPUs (Kepler, compute capability 3.5): Previously unable to start → 35 tps Fixes to cuBLAS compute-type selection for legacy gpu in mainline llama cpp to run on hardware that previously failed during initialization…

热度 45.1 / 100；排名与评分保留该期记录。