Compute#AI chips#Model inference
n8n tutorial: seven local models and a full workflow, offline on one DGX Spark, with measured throughput
DGX Spark local models: n8n's October 19 tutorial runs seven models on one box, 120 tok/s for one user. Its LiteLLM 1.84.0 floor lets affected versions through.
NVIDIA’s DGX Spark is a desktop AI workstation built around the GB10 chip. The author’s machine had 128GB of unified memory. On October 19, 2026, n8n’s blog published a tutorial on serving seven local models from one such box, behind a single LiteLLM endpoint, so its workflows never call a cloud API. The single-user speed it reports is 120 tokens per second, from one benchmark run on October 13.
Key points
- All seven models are Apache 2.0 or MIT licensed, which we checked model by model on Hugging Face. With everything loaded, the box uses 108GB of memory and has 13GB free.
- One user gets 120 tokens per second and eight users get 359 in total, with reasoning tokens counted. While an image generates, single-user chat drops from 122 to 38 tokens per second.
- The tutorial’s advice to run LiteLLM “1.84.0 or later” lets affected versions through. A September 30 advisory covers 1.91.0 up to but not including 1.100.4, and the author’s own 1.99.1 falls inside that range.
Background
The tutorial starts from a problem many small firms have. They want AI for emails, contracts and invoices, but cannot send customer data to a cloud API. NVIDIA describes the DGX Spark as a platform for building and running autonomous agents, with 273 GB/s of memory bandwidth and up to one petaflop of FP4 compute. The author tested the 128GB model. The 64GB version is sold only through OEM partners, and as of October 19 NVIDIA’s own page for it still said “Coming Soon”. We covered its launch plans in our DGX Spark 64GB story.
n8n’s self-hosted AI starter kit is built for one person running models on a laptop, and it uses Ollama. This tutorial is the team version: one box serves an office, with one model per job. The GPU and the operating system share a single memory pool, which Linux reports as 121GB.
The facts
These are the seven models that stay loaded. Licenses were checked against each model’s Hugging Face metadata.
| Job | Model | License |
|---|---|---|
| Chat, coding, vision, extraction | nvidia/Qwen3.6-35B-A3B-NVFP4 | Apache 2.0 |
| Embeddings | Qwen/Qwen3-Embedding-4B | Apache 2.0 |
| Reranking | Qwen/Qwen3-Reranker-0.6B | Apache 2.0 |
| Speech to text | openai/whisper-large-v3-turbo | MIT |
| Document OCR | zai-org/GLM-OCR | MIT |
| Image generation | Tongyi-MAI/Z-Image-Turbo | Apache 2.0 |
| Text to speech | hexgrad/Kokoro-82M | Apache 2.0 |
The chat run used vLLM 0.30.0 with pinned model commits. Per-user speed is a median.
| Concurrent users | Time to first token | Tokens/s per user | Tokens/s total |
|---|---|---|---|
| 1 | 0.11 s | 120 | 87 |
| 4 | 0.21 s | 68 | 242 |
| 8 | 0.29 s | 47 | 359 |
The single-user total sits below the per-user speed because one of five requests ran at 43 tokens per second. Other jobs, as the median of three runs:
| Task | Setup | Time |
|---|---|---|
| Text to speech | About 30 seconds of audio | 0.87 s |
| Speech to text | The same clip | 0.58 s |
| Image generation | 1024×1024, 8 steps | 12.2 s |
| Document OCR | One page | 1.80 s |
| Invoice extraction | Thinking off | 0.59 s |
| Embeddings | 32 passages | 0.18 s |
| Reranking | 32 candidates | 0.03 s |
Image generation at the server’s default step count runs about 48 steps and takes roughly 72 seconds per image. The author’s 8-step setting brings that to 12.2 seconds. The cold-start table puts the main model at 237 seconds to ready, while the prose says it was ready 311 seconds after docker run. A prompt of 8,537 tokens got its first token in 1.14 seconds. With all seven models loaded, the box used 108GB. The cold-start table lists six services and leaves out the embedding model.
Prices, from the reports below:
| Item | Price (USD) | When | Source |
|---|---|---|---|
| 128GB model, launch price | 3,999 | At launch | ServeTheHome, Tencent News |
| 128GB model, February 2026 | 4,699 | February 2026 | Tencent News, Wall Street CN |
| 128GB model, as reported October 2 | 6,950 | October 2026 | The Register, ServeTheHome |
| 64GB model, OEM channel | 4,999 and up | From October 23 | ServeTheHome, Tencent News |
A version correction
The tutorial’s fourth step says to run LiteLLM 1.84.0 or later. Its reasons: two PyPI releases were backdoored in March 2026, and 1.84.0 “fixed two authentication bypasses.” We checked the advisories that cover these versions, and three points need correcting.
- The March malware. Advisory GHSA-5mg7-485q-xm76 (March 25, critical) says litellm 1.82.7 and 1.82.8 were uploaded to PyPI with malware that harvests credentials and files. Anyone who installed and ran either version should rotate every credential in that environment. 1.84.0 is outside that range.
- The authentication bypasses. The two April advisories, GHSA-jjhc-v7c2-5hh6 and GHSA-69x8-hrgq-fjj8, are fixed in 1.83.0, not 1.84.0. Since 1.84.0 is higher, neither advisory makes the tutorial’s advice unsafe, but the attribution is wrong.
- The September privilege escalation. Advisory GHSA-7hp6-4w63-5g45 (September 30, critical, CVSS 9.9) describes escalation to proxy admin through reuse of the salt key. An authenticated internal user can reach proxy admin and run commands on the host through the MCP stdio endpoint. The advisory says versions after 1.91.0 are exploitable in the default configuration, and 1.87.0 through 1.90.x only when
EXPERIMENTAL_UI_LOGIN=trueis set. The affected range runs from 1.91.0 up to but not including 1.100.4, with separate ranges on the 1.101, 1.102, 1.103 and 1.104 release-candidate branches. The author’s 1.99.1 is in range.
The advisory’s fix is to upgrade. If you cannot upgrade yet, EXPERIMENTAL_UI_LOGIN=false closes the path, but the advisory warns that it also breaks CLI SSO and Claude Code gateway login. The branch ceilings are 1.100.4, 1.101.3, 1.102.2, 1.103.1 and 1.104.0rc2, and none of them falls inside an affected range. Pick a release outside every listed range rather than stopping at 1.84.0. We did not review every LiteLLM advisory.
What others say
Independent measurements land in the same range as n8n’s numbers, though the setups differ.
- ctala’s benchmark page (updated October 6) tested the same model on vLLM 0.31.0 with multi-token prediction at k=2 and a 128K context. It reports 102.3 tokens per second for one user and 240.0 for four, using about 56GB. The four-user total is close to n8n’s 242.
- LLMRequirements (June 3) measured 97.4 tokens per second for one request at a time, and about 322 tokens per second across eight concurrent streams. That figure is decode-only and uses prefix caching.
- SMF Clearinghouse (July 5) averaged 107.3 tokens per second across a mixed set of image, video and other tasks. Its serve command turned thinking off to match other models. n8n’s numbers count reasoning tokens, so the two are not directly comparable.
The skeptics have a point. Devashish’s blog (July 7) says the author tried the 35B-A3B model, called it “not a reasoning model” that “failed on these tasks,” and moved to a 27B model. That is one user’s experience with no public test set, and it describes the model differently from a tutorial that counts reasoning tokens. We cannot say which account is right. Tom’s Hardware’s review of the Mac Studio M5 Ultra (September 21) says its throughput is almost four times the DGX Spark’s across its tests. The review gives only that ratio, with no DGX Spark absolute figure, and it used a dense model, so the ratio cannot be converted into n8n’s numbers.
Our take
The most reusable part of the tutorial is its discipline. It pins each model to a Hugging Face commit, allows only a short list of licenses, gives every team its own LiteLLM virtual key, and tests end to end with one realistic workflow. Those habits carry over to another team’s stack more easily than the throughput figures do.
The figures themselves need a discount. They come from one run by the author, with reasoning tokens counted. The extraction result was checked on one clean, single-page invoice, and the author says scanned or photographed invoices need their own test.
One inference is ours, and we flag it. The seven-model stack uses 108GB, and the 64GB model has 64GB of memory, so the full stack will not fit on that machine. We drew that from the published numbers and did not test it.
Image generation is the sharpest trade-off. Chat and images share one GPU, so single-user chat fell to 38 tokens per second while images generated. A small team can queue image jobs or run them after hours. The author also suggests moving media work to a second box once images become a daily task.
Price is the other variable. The table above shows the 128GB model going from $3,999 to $6,950. The Register attributes the rise to memory prices. For the memory backdrop, see the memory shortage story. Two Chinese outlets report a $4,699 step in February 2026. Wall Street CN attributes it to “media reports.” The two English sources we read, The Register and ServeTheHome, do not mention it, so we treat it as unconfirmed.
How to try it
If you do not have a DGX Spark, the author suggests starting with the self-hosted AI starter kit. If you do, start with three checks:
- Print the vLLM version inside your image with
docker run --rm --entrypoint python3 <image> -c "import vllm; print(vllm.__version__)". The author’s first nightly image contained vLLM 0.19. It could not load Qwen3.6, and its error looked like a broken checkpoint. - Check your LiteLLM version against the advisories above. Do not stop at 1.84.0.
- Import the invoice workflow template the tutorial provides. It has six nodes, including OCR and a Qwen3.6 extraction step with thinking turned off. In the author’s single-page sample, that step took 0.59 seconds instead of 12.1.