2026-10-12 · America/Los_Angeles · 社区动态 · #18

Serving GLM-5.3-Flash on 8×H100 in 37s with vLLM snapshots

Hi r/LLMDevs, I built warmctl, an open-source tool that takes a snapshot of a fully warmed-up vLLM engine and restores it in seconds on a different host, with multi-GPU support. So you can actually scale to zero and come back up in seconds, skipping the 10-minute cold start. Modal and Inferize or RunPod FlashBoot do something similar, but they’re hosted. This one is open source and runs on your own GPUs. Repo: https://github.com/TensorRails/warmctl How it works in tldr: vLLM sleep mode offloads weights. CRIU checkpoints the process. CUDA checkpoint API saves GPU state. cuInterpose handles…

热度 45.4 / 100；排名与评分保留该期记录。