DeepSeek Sparse Attention Cuts Long-Context Cost

When DeepSeek shipped V3.2-Exp on September 29, 2025, it cut its API prices by more than 50% the same day and credited one architectural change: sparse attention. Six months later, V4 made a 1M-token context the default tier at no premium. Both moves trace back to DeepSeek Sparse Attention (DSA), a mechanism that breaks the quadratic cost of attention at long context without the accuracy loss that sank earlier sparse methods. For anyone paying per token to run 128K-plus context, DSA changes which workloads are affordable.

Key takeaways

DSA uses a two-stage design: a lightweight "lightning indexer" scores every prior token, then standard attention runs over only the top 2,048 tokens the indexer selects.
The change drops attention complexity from O(L squared) toward O(L times k) at prefill and to near-constant O(k) at decode, since k stays fixed at 2,048 no matter how long the context grows.
DeepSeek reports V3.2-Exp performs on par with the dense V3.1-Terminus; independent RULER testing shows roughly a 1% accuracy drop at 128K and about 2% at 512K.
Spheron's 8x H200 benchmarks put time-to-first-token near 2x faster at 128K and 3.2x faster at 512K versus dense attention.
The indexer's overhead makes DSA slower than dense attention below roughly 40K to 64K tokens, so it is a long-context optimization, not a universal one.
DeepSeek V4 (April 24, 2026) ships DSA as the default and prices a 1M-token context at baseline rates, a real shift in long-context economics.

What Is DeepSeek Sparse Attention, and What Problem Does It Solve?

DSA solves the quadratic wall in transformer attention: as context grows, the work to attend every token to every other token scales with the square of sequence length, so a 128K prompt costs roughly sixteen times the attention compute of a 32K one. That cost lands twice, once in prefill when the model reads the prompt, and again on every decode step as it generates each new token. Long-context requests dominate both latency and the GPU bill for exactly this reason.

DeepSeek's answer is selective attention. Instead of having each query attend to all L previous tokens, DSA attends to only the k most relevant ones, with k fixed at 2,048. The technical report frames the reduction as moving from O(L squared) toward O(L times k). At 128K tokens, k of 2,048 means each query reads under 2% of the context.

DSA sits on top of Multi-head Latent Attention (MLA), the KV-cache compression DeepSeek introduced in V2 that already shrank cache size by roughly an order of magnitude. DSA doesn't replace MLA; it decides which of those compressed entries actually get read. The two compose, which is why the savings stack rather than compete. If you have not internalized why the cache is the cost center here, my note on why KV cache is the biggest lever in inference cost covers the ground DSA builds on.

How Does the Lightning Indexer Pick Which Tokens Matter?

A component DeepSeek calls the lightning indexer scores every cached token against the current query, and the 2,048 highest scorers get passed to full attention. The indexer is deliberately cheap. In the tensoreconomics teardown of the mechanism, it runs 64 small heads that each compute a dot product against a single shared key, apply ReLU so only positive relevance counts, and sum the result. Because it uses one shared key and low-dimensional heads (64 dimensions against MLA's 512), scoring the whole sequence stays affordable even when the sequence is very long.

The design matters because it sidesteps the failure mode of fixed-pattern sparsity. Sliding-window attention keeps only the most recent N tokens and structurally forgets anything older, so it fails retrieval tasks where the answer sits early in the prompt. DSA's indexer selects from anywhere in the context, so a relevant token 100K positions back still gets attended if it scores high enough. That is the difference between cheap-and-blind and cheap-and-targeted, and it is why DSA holds retrieval accuracy where a sliding window would not.

What Does DSA Actually Cost in Compute and Memory?

Decode time goes nearly flat and prefill scales close to linearly, because the expensive full-attention step now operates over a fixed 2,048 tokens instead of the whole sequence. The two phases split differently. At decode, the indexer costs O(S) to scan the sequence while attention costs a constant O(k), so per-token generation barely slows as context grows. At prefill, the indexer itself is the quadratic term, but it is cheap enough per operation that the net prefill cost drops sharply at long context.

The published and measured numbers line up on that shape:

Context length	Attention FLOPs cut vs dense	Time-to-first-token speedup (8x H200)	RULER accuracy drop
32K	Near break-even (below crossover)	1.3x	Negligible
128K	~98%	2.0x	~1%
512K	~99%	3.2x	~2%

The FLOPs and latency figures above come from Spheron's deployment benchmarks; the accuracy figures are from RULER testing they cite. On the memory side, the tensoreconomics analysis reports the per-decode-step KV read at 131K context falling from about 5.2 GB under dense MLA to about 1.1 GB under DSA, because the indexer reads all tokens in a compact FP8 form (roughly 132 bytes each) while full attention loads only the 2,048 selected tokens in full. Storage per token doesn't change; the read footprint does, and read bandwidth is what bounds decode.

Translated to price, Spheron put a 128K workload on 8x H200 spot capacity at about $8.56 per million output tokens with DSA against about $17.51 without, close to the 2x reduction that mirrors DeepSeek's own 50%-plus API cut. Those savings compound with the serving techniques I have covered separately, including prefill-decode disaggregation and speculative decoding, since DSA attacks attention cost specifically and leaves those levers intact.

Does Sparse Attention Hurt Accuracy?

Barely, on the evidence published so far. DeepSeek reports V3.2-Exp lands on par with the dense V3.1-Terminus across its benchmark suite, and it deliberately trained the two on the same data to keep the comparison honest. Independent RULER testing cited by Spheron shows roughly a 1% drop at 128K and about 2% at 512K, with needle-in-a-haystack retrieval nearly identical to dense attention.

The accuracy holds because most attention is already sparse in practice. Studies of trained transformers repeatedly find that a small fraction of tokens carry most of the attention mass for any given query, so selecting the top 2,048 discards mostly near-zero weights. DeepSeek made the same bet earlier with Native Sparse Attention, a trainable and hardware-aligned pattern that fed into DSA's design.

The caveat a senior engineer should hold onto: RULER and needle tests probe retrieval more than they probe dense synthesis. The open question is whether 2,048 selected tokens suffice for genuinely multi-hop reasoning over a long input, where many tokens matter at once and no single one dominates the score. That is precisely the workload the current benchmarks under-test.

When Does DSA Beat Dense Attention, and When Doesn't It?

Above roughly 40K to 64K tokens it wins, and below that it loses. The indexer adds fixed overhead on every step, so at short context that overhead outweighs the savings and dense attention comes out ahead. The tensoreconomics analysis puts the crossover near 40K; Spheron's deployment testing puts it near 64K. Either way, running DSA for 4K-token chat turns would cost you speed, not save it.

Sparse attention is also not the only way to reach long context, and the alternatives solve different problems:

Technique	How it scales context	Best fit	Main limitation
Dense (full) attention	O(L squared)	Short-to-mid context where accuracy is paramount	Quadratic cost past ~32K tokens
Sliding-window (e.g., 4K)	Linear in window size	Streaming and recent-context tasks	Structurally forgets older tokens; fails long-range retrieval
DeepSeek Sparse Attention	Near-linear prefill, near-constant decode	128K to 512K, single-node, content-adaptive retrieval	Indexer overhead below ~40K to 64K; only a few models ship it
Ring attention	Distributes sequence across GPUs	1M-plus tokens across a cluster	Needs multi-GPU coordination and cross-device communication

They are not mutually exclusive. For the 512K-plus range, the practical pattern is DSA on each device combined with ring attention across devices, so the sparse selection cuts per-GPU cost while the ring spreads what remains.

What Does DeepSeek V4 Signal About Long-Context Economics?

It signals that million-token context is becoming a default, not a premium tier. DeepSeek V4, released April 24, 2026, ships DSA as the production default, scales the MoE to 1.6T total parameters with 49B active per token (about a 3% activation rate), and, per analyst coverage, makes 1M context the baseline across web, app, and API with no surcharge. At 1M tokens that same coverage estimates DSA's advantage over dense attention at roughly 500x.

The pricing pattern around these releases is the part worth planning around. Stanford HAI's AI Index has documented that competitive open-weight releases pull closed-API prices down within weeks, and DeepSeek's own 50%-plus cut at the V3.2-Exp launch fits that record. If you are modeling long-context features on today's frontier-API prices, DSA-class efficiency is a reason to expect those prices to keep sliding, which changes the build-versus-buy math for retrieval-heavy and agentic workloads and for how aggressively you route between models.

What Should You Watch Before Building on It?

Three constraints deserve attention before you commit a roadmap to it.

It is baked into the weights, not a serving flag. DSA is trained into the model, so you cannot bolt it onto Llama or Qwen without retraining. Current teardowns show only DeepSeek (V3.2 onward) and Zhipu's GLM line shipping it, which makes adoption a model-selection decision, not a tuning knob.
Kernel and framework support lags the model. DeepSeek open-sourced TileLang and CUDA kernels, but coverage across vLLM and SGLang and across non-Hopper hardware is still catching up. Deployment flags such as block size and chunked prefill are hardware-specific, so plan for tuning rather than a drop-in.
The reassuring accuracy numbers are retrieval-shaped. They come from RULER and needle-in-haystack style tests. Validate on your own long-context task, especially anything that demands dense multi-hop reasoning over the full input, before you trust the 1% to 2% figure.

One more honest flag: the sharpest cost and latency figures in this piece come from a GPU-cloud vendor with a direct interest in long-context demand. Treat them as directional and measure on your own stack. If long context is central to what you are building, pair this with my note on serving open models with vLLM, where the throughput trade-offs DSA shifts show up in practice.