Unsloth Quantizes DeepSeek-V4.1-Flash-1017: Engram Table Offloaded to SSD, Runs on 64GB RAM (Slowly)
Unsloth’s October 30 GGUFs offload DeepSeek’s 196B Engram table to SSD; official guidance says 128GB RAM works and 64GB runs slower.
On October 30, Unsloth shipped Dynamic V3.1 GGUF quantizations of DeepSeek-V4.1-Flash-1017, the MIT-licensed multimodal model with a 1M-token context that DeepSeek updated ten days earlier. The headline number — 748 billion parameters — decomposes, per Unsloth’s guide, into a 552B backbone plus a 196B Engram conditional-memory layer, and Engram is an n-gram lookup table that can live entirely on an SSD, paged in on demand through mmap. The result is a monster: the lossless UD-Q8_K_XL occupies 508.4GB on disk and even Q4 comes to 500.7GB, yet official guidance says 128GB of RAM is enough to run it (64GB works “slower”). The hardware floor didn’t disappear; it moved from RAM to disk and patience.
Key points
- Unsloth’s Dynamic V3.1 GGUFs span 1-bit to 8-bit (from UD-IQ1_S at 233.0GB), published October 30 with more than 420,000 cumulative downloads as of November 2.
- The Engram table — 202.8GB at MXFP8, 110.6GB at IQ4_NL — stays on SSD with rows cached on demand; the official hardware table lists 315GB RAM+VRAM for 8-bit and 128–145GB for 1-bit.
- The model’s HN thread (959 points, 726 comments) found its best-rated joke in the model card calling a double-digit benchmark jump “a minor update.”
Background
DeepSeek-V4.1-Flash launched September 10; 1017 is the October 17 refresh, which the model card describes as improving reasoning, coding, agentic and visual benchmarks (Terminal-Bench 4.0 rose from 31.2 to 42.3, Automation-Bench from 54.8 to 60.5, with GLM-5.3, GPT-5.6 Sol and Opus 5 in the comparison table). We covered the original release and the HN reaction to its benchmarks in September. What makes this quantization cycle different is not compression — Q4 is still 500.7GB — but reclassifying Engram, a table of up to 4-gram entries (config-verified: two layers of ~384M entries each over a 16M-vocab compressed embedding), from “must fit in memory” to “page in on demand.”
The facts
| Item | Value | Source |
|---|---|---|
| Parameter split | 552B backbone + 196B Engram | Unsloth guide |
| UD-Q8_K_XL (lossless) | 508.4GB | Unsloth guide, verified against HF file tree |
| UD-Q4_K_XL | 500.7GB (202.8GB of it the Engram shard) | Same |
| UD-IQ1_S (smallest) | 233.0GB including a 110.6GB IQ4_NL Engram | Unsloth guide |
| RAM requirement (RAM+VRAM) | 128–145GB for 1-bit; 315GB for 8-bit; 64GB runs “slower” | Official hardware table |
| Context | 1M tokens, max_tokens ≥ 256K recommended | Official model card |
| Swappable Engram precision | MXFP8 / Q6_K 161.3 / Q5_K 135.2 / IQ4_NL 110.6GB | Mix-n-match guide |
The quants follow Unsloth Dynamic 3.1: Q4 and Q8 keep the MXFP4 expert weights and the same Engram table; Q4 stores most non-expert weights at Q8_0 and Q8 at BF16 — which is why Q4 and Q8 differ by only 8GB. The bits you’d shave are worth nearly nothing, so it doesn’t compress. Inference requires a specific llama.cpp PR or Unsloth Desktop; Unsloth claims up to 3x faster inference than other CPU/ngram-focused repos, and an 8GB DSpark speculative-decoding module can speed decoding further.
What others say
On the 959-point HN thread, the top comment is @freakynit deploying the Padmé meme: “Those benchmarks can’t possibly be real, right? Right?” @jononor voiced the local-hardware crowd’s position — “I do wish that we will see some smaller ‘flash’ models again though - running this one locally is tough (very costly)” — while [@Pragmata] did the math: a pair of DGX Spark units would run it, about $8,000 in hardware at launch pricing. On the Engram approach itself, Unsloth’s guide leads with KLD-convergence data (the Q8 quant trails the reference by 0.3 percentage points of top-1 agreement on wiki1100), an unusual degree of quantization-quality disclosure. What the official data does not cover is whether SSD paging sustains throughput in long-context, high-hit-rate sessions — that awaits third-party measurement.
For an ecosystem contrast: NVIDIA’s official NVFP4 quantization, also released October 30, targets data-center GPUs, while Unsloth’s GGUFs aim at consumer memory — two ends of the same problem. For how similar mmap-style tricks squeeze big models into constrained hardware, revisit our coverage of the GPU tensor-core unlock and projects like Podroid that fit systems into spare resources; on the enterprise side, wiring open models into production gateways is the territory of agentgateway.
Our take
For the local-LLM community, the practical read is that the bottleneck has moved: the budget question is no longer RAM or VRAM but disk capacity and random-read speed, and buying NVMe beats buying DIMMs. For enterprise deployments, a 128GB Mac Studio running an MIT-licensed flagship without data leaving the building is a new price anchor for compliance-sensitive setups. The honest catch: there is no number for the distance between 64GB “runs” and 64GB “runs comfortably,” and Engram’s paged throughput under long sessions has only vendor claims behind it. The 508GB download and disk footprint also remain a one-time wall for most people, and lower-bit quants demand a larger reasoning budget for the model’s variable-length thinking. Worth watching: when the llama.cpp PR lands in mainline, and whether the vLLM ecosystem picks up Engram offloading.