How to Fix vLLM CUDA Out of Memory on Model Startup

A vLLM server that dies the instant it loads a model is almost never a broken install. It's an accounting problem. The model weights, the activation and CUDA graph buffers, and the KV cache all have to fit inside one memory budget, and by default vLLM claims 92% of the card and carves the cache out of whatever is left. The quick fix for the common case is to cap the context you actually serve with --max-model-len, and if the card still has room, hand vLLM a bigger slice with --gpu-memory-utilization. First, read the error, because two different failures look almost identical.

What You Need Before You Start

This assumes vLLM already installs and can, in principle, load your model. If it cannot serve anything yet, get a clean baseline first with the guide to serving an open LLM with vLLM, then come back to tune memory.

• A working vllm install and a model that at least downloads.
• nvidia-smi to read total and free VRAM on the target card.
• Your real workload numbers: the longest context you serve, and how many requests run at once.

How Do You Clear a vLLM Startup Out-of-Memory Failure?

Work top to bottom and stop the moment the server comes up. Each step trades a little capability or precision for memory, so apply the cheapest one that clears your specific error first.

1. Read the error and classify it. A bare torch.OutOfMemoryError: CUDA out of memory during weight loading means the model does not fit in the budget at all. A ValueError means the weights fit but there is no room for a full context. As shown in this vLLM issue thread, it reads: The model's max seq len (4096) is larger than the maximum number of tokens that can be stored in KV cache (3664). Try increasing gpu_memory_utilization or decreasing max_model_len when initializing the engine. Steps 2 to 4 clear the KV cache case; step 5 clears the weight case.
2. Cap the context length to what you serve. KV cache demand scales linearly with sequence length, so context you never use is memory burned for nothing. Set the ceiling to your real longest prompt plus completion:

   vllm serve meta-llama/Llama-3.1-8B-Instruct \
     --max-model-len 8192

   If the error named a KV cache capacity, set the length at or below that number.
3. Give vLLM a larger slice of the GPU, in small steps. The default gpu_memory_utilization is 0.92, a fraction of total memory (not free memory) reserved for this instance. If nothing else runs on the card, nudge it up:

   vllm serve meta-llama/Llama-3.1-8B-Instruct \
     --max-model-len 8192 \
     --gpu-memory-utilization 0.95

   Raise it a few points at a time. Push it too far and you starve CUDA graphs and the OS, which swaps the KV cache error for a hard CUDA OOM.
4. Halve the KV cache with an fp8 cache dtype. On Ada and Hopper-class GPUs you can store keys and values in 8-bit and roughly double the tokens the same memory holds:

   vllm serve meta-llama/Llama-3.1-8B-Instruct \
     --max-model-len 8192 \
     --kv-cache-dtype fp8

   The default is auto (the model's own dtype). fp8 costs a little accuracy, so measure on your own eval before you ship it.
5. If the weights themselves don't fit, shrink or split them. When step 1 showed a CUDA OOM during loading, no cache tuning helps: the model is too big for one card's budget. The official memory conservation guide lists the levers. Load a quantized checkpoint (--quantization awq or a pre-quantized repo), split across GPUs with --tensor-parallel-size 2, or spill weights to host RAM with --cpu-offload-gb 10 (slower, but it starts).
6. Confirm it started and left headroom. A server that boots with almost no spare cache will stall under load. Send one request, then watch the cache gauge:

   curl http://localhost:8000/metrics | grep kv_cache_usage

   If vllm:kv_cache_usage_perc sits near 1.0 with light traffic, lower --max-num-seqs or --max-model-len further.

Why Does vLLM Run Out of Memory Before Serving a Single Token?

vLLM sizes everything up front so it never has to allocate mid-request. At startup it reserves gpu_memory_utilization times total GPU memory, loads the weights, runs a profiling forward pass, then hands the entire remainder to the paged KV cache. That last number is the one that bites: if the leftover can't hold even one sequence of max_model_len tokens, the engine refuses to start rather than crash later.

So the KV cache, not the weights, is usually what you're short on, and it dominates memory planning for any long-context or high-concurrency deployment. For the deeper economics of that trade, see why the KV cache is the biggest lever in inference cost.

How Big Is the KV Cache Per Token?

Per token, the cache stores a key and a value for every layer and every attention head: roughly 2 * num_layers * num_kv_heads * head_dim * bytes_per_element bytes. Total capacity is that figure times max_model_len times how many sequences run at once. Two of those multipliers are yours to set at launch, --max-model-len and --max-num-seqs, and shrinking either shrinks the cache in direct proportion.

Which Memory Knob Should You Reach For?

Each flag buys memory in a different place. Match the flag to the failure you actually have.

Troubleshooting the Failures You'll Still Hit

• It started, but requests stall and the log warns about preemption. vLLM V1 defaults to RECOMPUTE preemption when KV cache space runs out under load, and the warning names a sequence group being preempted. Lower --max-num-seqs or raise the memory budget so more sequences fit at once.
• CUDA out of memory appears only after minutes of traffic. Another process grabbed VRAM after vLLM measured free memory. Because gpu_memory_utilization is a fraction of total, not free, memory, a second job on the card breaks that assumption. Pin the server to its own device with CUDA_VISIBLE_DEVICES.
• Lowering max_model_len changed nothing. If you also set --max-num-batched-tokens below max_model_len with chunked prefill turned off, vLLM cannot schedule a full prompt. Leave chunked prefill on (the default) or raise the batched-token limit above your context length.
• fp8 cache was rejected. Your GPU or build lacks fp8 support. Fall back to cutting max_model_len or splitting the model across cards.

What to Do Next

Once the server boots cleanly, tune for your traffic rather than just survival. The same optimization guide covers chunked prefill and max_num_batched_tokens, the two knobs that trade latency against throughput once memory is no longer the blocker. Load-test at your real concurrency and keep vllm:kv_cache_usage_perc under 1.0 with headroom to spare, so a burst of long requests doesn't tip you back into preemption.

Sources

• vLLM, Conserving Memory
• vLLM, Engine Arguments
• vLLM, Optimization and Tuning
• vLLM GitHub Issue 2418, KV cache ValueError