AI Resonance · 阅读最新日报 · AI 入门推荐

2026-11-17 · America/Los_Angeles · 社区动态 · #12

Rotate K/V read order for better L2 cache hit rates by gaugarg-nv · Pull Request #32407 · ggml-org/llama.cpp

Significant prompt processing speedup with long contexts gpu_info model_type n_prompt n_gen n_depth Master (t/s) PR (t/s) Speed-up NVIDIA GeForce RTX 5090 qwen35 27B Q4_K - Medium 512 0 131072 1290.10833 1822.36027 1.41 NVIDIA GeForce RTX 5090 qwen35 27B Q4_K - Medium 512 0 65536 2030.31282 2527.97409 1.25 NVIDIA GeForce RTX 5090 qwen35 27B Q4_K - Medium 512 0 32768 2931.96118 3142.20539 1.07 NVIDIA GeForce RTX 5090 qwen35 27B Q4_K - Medium 512 0 16384 3562.18311 3573.59336 1.00 NVIDIA GeForce RTX 5090 qwen35 27B Q4_K - Medium 512 0 8192 3854.83184 3850.97547 1.00 NVIDIA GeForce RTX 5090…

Rotate K/V read order for better L2 cache hit rates by gaugarg-nv · Pull Request #32407 · ggml-org/llama.cpp

热度 42.1 / 100;排名与评分保留该期记录。