DeepSeek's new 304B agentic model now runs on a single 128GB workstation

DeepSeek shipped a 304-billion-parameter model on July 31 built for agent work. Salvatore Sanfilippo had the released weights running on a 128GB desktop by Saturday morning, past 20 tokens a second, with no lossy quantization step in between.

That pairing is the story. Models with open weights have sat next to the frontier for months, and the Kimi K3 wave two weeks ago made that obvious. What’s new is the shape of the file. A model DeepSeek tuned specifically for agent work scored 50 on Artificial Analysis’s Intelligence Index against GPT-5.6 Luna’s 51, and you can now run it from the weights as released instead of from a 2-bit compression of them.

Why “lossless” is the load-bearing word

Start with what DeepSeek actually published. The checkpoint runs about 167GB on Hugging Face and carries 304 billion parameters, per Simon Willison’s notes on the release. The model card puts the license at MIT and the context window at 1 million tokens, and sells the update as bringing “substantially enhanced agentic capabilities”. The weights ship in FP4.

The agent claim has numbers behind it. That model card lists 82.7 on Terminal Bench 2.1, 54.4 on DeepSWE and 68.7 on DSBench-FullStack, and states that the checkpoint “outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count”. Artificial Analysis tracks the same jump from a different angle, scoring an Elo of 1559 on GDPval-AA v2 against 1189 for the April build. Those are tool-use and repo-level tasks rather than trivia. A model that holds up on them is one you’d point at a codebase and leave running.

That last part matters more than it sounds. DeepSeek never published a BF16 copy, because the expert tensors were trained with FP4 quantization-aware training, a detail the community GGUF conversion card spells out. Every local build until now has therefore been a quantization of a quantization: take FP4 weights, crush the routed experts down to a 2-bit and 4-bit blend, and hope the router still behaves. That card’s file lands at 97.6GB by putting most expert layers in IQ2_XXS and Q2_K while protecting the attention projections and the router, on the logic that “Everything every token passes through stays high precision”.

MXFP4 is how you skip the crushing step. It’s the Open Compute Project’s microscaling format: 32 weights share one 8-bit power-of-two scale, each weight gets two exponent bits and one mantissa bit, and the encoding averages about 4.25 bits per value. It is also the format DeepSeek’s routed experts already sit in. Sanfilippo published a GGUF on his Hugging Face account that leaves those experts bit-exact and keeps attention, shared experts and output at Q8 or F16, which puts the file at 156GB according to the issue thread tracking it.

Sanfilippo, the Redis creator who posts as antirez, described the result on Bluesky on Saturday morning: “It rocks even with SSD streaming in 128GB systems at > 20 t/s in case you want to try the actual DS4F weights released without any quantization.” The branch that runs it is called ds4f-mxfp4.

What 167GB asks of your desk

A 156GB file does not fit in 128GB of RAM. That’s the whole trick. DwarfStar, Sanfilippo’s C inference engine (the project also answers to ds4), keeps the non-routed weights resident and holds the routed MoE experts in a cache, pulling them off the GGUF on a miss. You set the budget with a flag like --ssd-streaming-cache-experts 32GB. NVMe read latency becomes part of your token rate, which is why he quotes past 20 tokens a second rather than the 34.3 that the project’s benchmark table reports for a fully resident 2-bit build on an M5 Max with 128GB of unified memory.

The published baselines are worth reading before you buy anything. DwarfStar’s requirements start at 96GB of Apple Silicon memory for 2-bit builds and 256GB for 4-bit, with a 512GB Mac Studio for the Pro variants. Generation on that M5 Max falls from 34.3 to 25.9 tokens a second once the prompt gets long. A DGX Spark with 128GB prefills 7,047 tokens at 343.8 tokens a second and then generates at 13.8. Prefill-heavy, output-light is exactly the shape of an agent loop, so those two numbers matter more than any single headline figure. At the cheaper end, the fully local coding agent stack we walked through in June still runs 30B-class models on a 32GB Mac.

Context is its own ceiling. DeepSeek advertises 1 million tokens, while the ds4 GGUF card documents context windows up to 100,000 and pushes past that with disk-based KV caching. DwarfStar also parks long prefixes on the SSD and resumes them by prompt hash, which is genuinely useful when an agent replays the same system prompt every run. It is not the hosted window.

One caveat before you clone anything. The MXFP4 file doesn’t load on ds4’s main branch at all: the routed expert tensors carry GGUF type 39, and the runtime aborts with “tensor has type 39 (unknown), expected a routed expert quant type”, per the open issue. Same on the laguna-s2.1 and glm5.2 branches. Only ds4f-mxfp4 has the decode path. This is a days-old branch of a one-person engine, not a supported product.

Where the hosted copy still wins

Price, first. Artificial Analysis lists the API at $0.14 per million input tokens and $0.28 per million output, with cache hits discounted 98% to $0.0028. The same page puts the new checkpoint at 50 on its Intelligence Index, ten points above April’s V4 Flash, six above V4 Pro, one behind GPT-5.6 Luna. Beating $0.14 per million on your own hardware means burning an enormous number of tokens before a 128GB workstation pays for itself.

Then there’s the reasoning-effort problem. The model card exposes low, high and max levels, and Willison found the default disappointing on his pelican-on-a-bicycle test until he re-ran it with reasoning_effort high, at which point the drawing improved sharply. Artificial Analysis reached its score of 50 at max effort, spending roughly 206 million output tokens across the eval suite. On a hosted endpoint that’s a line item. On your desk at 20 tokens a second, it’s wall-clock time you sit and watch.

The hosted path is also already current. Vercel flipped its AI Gateway to the new weights on July 31 with no code change for anyone calling deepseek/deepseek-v4-flash, noting that DeepSeek itself was the only provider serving them at first and that others were expected within a week. The changelog credits the update with a Terminal-Bench score of 82.7, up 25.8 points from the April preview’s 56.9. That’s the benchmark closest to what an agent actually does all day.

What this means for you

My read: local agent work is now practical for a narrow set of reasons, and saving money is not one of them. If your code or data can’t leave the network, or you want a long-running agent that keeps working when someone else’s API doesn’t, this is the first checkpoint where going local costs you throughput rather than intelligence. You’re running the numbers DeepSeek released. A 2-bit repack never let you say that.

If the bill is your motive, the arithmetic is unkind, and it keeps getting worse as inference prices fall. The gray market in relayed API keys exists because per-token pricing is already cheap enough to be worth stealing rather than replicating. Watch two things this month: whether MXFP4 decode lands on ds4’s main branch, and whether the other providers ship the 0731 weights on that one-week timeline. Until the first happens, running the real weights means running a branch that one person maintains.