Models#Benchmark#Model inference
Ant open-sources Ling-3.1-flash weights (MIT): ~560B MoE, but the served context is 256K today
Ant put Ling-3.1-flash weights on Hugging Face on Oct 13 under MIT, ahead of its stated plan; config and docs cap served context at 256K, with 1M unopened.
On October 13 Ant’s Bailing team published the weights for Ling-3.1-flash to the Hugging Face repository inclusionAI/Ling-3.1-flash under MIT, in 54 safetensors shards. The model card’s architecture table lists “63 KDA + 9 Gated MLA (7:1)” alongside 4 dense layers and 512 routed experts, 8 active per token plus one shared expert, while config.json reports num_hidden_layers as 72. The figure worth tracking ships alongside the weights: config.json sets max_position_embeddings to 262144, and the vendor’s own documentation states the model’s context window caps at 1M while the official service currently exposes 256K. The advertised 1M is not something you can use today.
Key points
- The weights landed on October 13, not October 15. The Hugging Face API reports
createdAtas 2026-10-13T11:16:41Z withgated=false; the 54 shards sum to atotal_sizeof 573,753,281,152 bytes. Chinese coverage named October 15 as the open-source date, two days after the repository was created. - 1M is the ceiling, 256K is the product.
config.jsonsetsmax_position_embeddings=262144and the SGLang deployment docs state a context limit of 262,144 tokens. Bailing’s own September 30 post reads: “免费体验期间模型服务长度为 256K；免费体验期结束并转为付费服务后，计划开放 1M 上下文” — 256K during the free period, 1M planned once billing starts. - Independent measurements don’t match the banner. Artificial Analysis scores the model at an intelligence index of 41, 5th of 176 models, with 213.2 tokens/s output and 1.79s to first token. The same page still labels the model proprietary with weights “not publicly available,” a judgment the October 13 upload overturned.
Background
Ling-3.1-flash arrived as a service first, on September 30. Bailing’s official WeChat account posted “Ling-3.1-flash：面向真实世界任务的模型升级” that evening, covering the architecture, office and software-engineering tasks, and a two-week free trial. On open-sourcing, the original sequencing reads: “免费体验期结束并转为付费服务后，计划开放 1M 上下文。届时，团队也计划同步开源” — the 1M context and the open-source release were both scheduled for the moment the free period ended. No weights shipped on that date.
In the two weeks that followed it became a fixture of long-horizon coding conversations. OrcaRouter, an AI routing service, published the most complete read on October 12, and framed it as a study in announced versus shipped: the official API (api.ant-ling.com, requiring an Alipay account and activation) and Ling Studio worked, with a 2 req/s rate limit and a 90-second non-streaming timeout, but there was no repository under the inclusionAI organisation on Hugging Face — in its words, “no model card, no licence text, and no checkpoint file.” The post records its own check as “a check against our live model list on 2026-10-13 returns no Ling or inclusionAI entry”: even on October 13 the repository was not there. The weights then landed later that day (HF createdAt 2026-10-13T11:16:41Z), which means Ant moved ahead of its own stated order — it published the weights without waiting for the 1M window.
The competitive tempo is real. Zhipu put GLM-5.3-Highspeed into randomized grey-box testing in Zcode on October 12, and MiniMax shipped its M3.1-Flash preview with 1M context to subscribers only at the end of September. The Chinese Flash-tier race now turns on long context, output speed and deployment cost; open weights are one item on that list.
The facts
What the Hugging Face repository itself says (checked October 15):
| Item | Value |
|---|---|
| Repository | inclusionAI/Ling-3.1-flash |
| Created | 2026-10-13T11:16:41Z |
| License | license:mit |
| Weight shards | 54 safetensors files; index total_size 573,753,281,152 bytes |
| Context | max_position_embeddings = 262144 |
| Architecture | BailingMoeV3ForCausalLM, num_hidden_layers=72; the model card lists “63 KDA + 9 Gated MLA (7:1)” plus 4 dense layers |
| Experts | 512 routed, 8 active per token, plus 1 shared expert |
| Vocabulary | 157,184 |
| Quantization | quant_method: fp8; the SGLang docs describe FP8 E4M3 weights in 128×128 blocks with dynamic activation quantization |
| Access | gated=false, no request form |
The layer count has two framings and neither is an equation. The model card and the WeChat post both state “63 KDA + 9 Gated MLA (7:1)” (the post’s phrasing: “按照 7 层 KDA 配 1 层 Gated MLA 的方式组成混合结构”), but num_hidden_layers=72 does not equal 63 plus 9. Counting the layers in model.safetensors.index.json directly: layers 0–3 are dense (they carry mlp.down_proj and no KDA or MLA projection), 63 layers carry the KDA k_conv1d, the Gated MLA layers with kv_a_proj_with_mqa are 7, 15, 23, 31, 39, 47, 55, 63 and 71, and layer 72 is the MTP/nextn output layer, which also carries an MLA projection. That sums to 4 + 63 + 9 + 1. Both framings come from primary material; adding 63 and 9 to reach 72 is the mistake to avoid.
Parameters carry two framings of their own: the model card and vendor documentation say about 560B total and about 25B active per token, while the index reports a total_size of 573,753,281,152 bytes (about 573.8 GB in decimal, about 534.3 GiB). Those are one number in two units — a parameter count and a weight-byte count — not two independent sources. Neither the model card, config.json, nor the SGLang docs publish any other weight-size figure in GB.
The deployment recipe is unusually complete for a day-0 release. SGLang: 8×H200 with --tp 8 --ep 8, or 4×GB300 with --tp 4 --ep 4, both with --speculative-algorithm NEXTN to engage MTP speculative decoding; vLLM uses --speculative-config '{"method":"mtp","num_speculative_tokens":3}'. The cookbook records 251.2 tokens/s at batch size 1 on H200 TP8+EP8 with NEXTN and the decode gates enabled, 180.0 tokens/s on GB300, and speculative acceptance lengths of 2.61 on random tokens against 2.13 on real-text probes.
Artificial Analysis (page viewed October 15) provides the independent third-party numbers: intelligence index 41, ranked 5th of 176 models; output 213.2 tokens/s against a tier median of 109.2; time to first token 1.79s against a median of 1.81s. Its pricing line reads $0.30 per million input tokens and $0.90 per million output tokens, with an 80% cache discount, and a blended $0.19/M derived from a 7:2:1 cache-hit/input/output ratio — a third-party construction, not a listed rate. On OpenRouter, inclusionai/ling-3.1-flash lists a context of 262,144, served by Novita at $0.075/M prompt, $0.225/M completion and $0.015/M cached reads (a 0.75 discount), which does not match Artificial Analysis’s input/output pair.
The vendor’s own long-task claims, all first-party and not run head-to-head against competitors on one harness: a Lua-to-x86-64-ELF compiler passing 178 of 182 tests (about 97.8%, against 99.0% for Claude Fable 5 and 95.9% for GLM-5.3-Flash), a 1.432× speedup on Pyright 1.1.400 (against 1.502× and 1.290×), and an 8.015× Rust reimplementation of a C image library (against 1.396× for GLM-5.3-Flash and 27.310× for Claude Fable 5).
What others say
The most useful line in OrcaRouter’s October 12 post is how it splits the context claim: the window caps at 1M, the official service hands out 256K, and the recommendation is to size for 256K and treat 1M as an undated roadmap item. It also documents interface-level rough edges — temperature and top_p default to 1, the thinking toggle is documented only under Ling-3.0-flash, and the model is absent from the API reference’s model list and the changelog. Artificial Analysis, by contrast, still files the model as proprietary with weights unavailable and records its context as 1M; both of those judgments expired on October 13, though the speed and intelligence measurements it took remain usable.
Chinese reporting is looser with the details. The InfoQ article names Ling-3.0-flash in the headline and Ling-3.1-flash in the body, and its text says only “最高支持 1M 上下文”, without the 256K qualifier. The page carries no verifiable publish-date metadata (no datePublished, no structured data, and the body references “10 月 15 日” without a year), so it is cited here on its text alone rather than on a publication date. That same article carries the MXFP4 claim — MXFP4 on routed experts with block-FP8 kept on dense layers, weight footprint dropping from about 534GB to 292GB, and Hopper service going from 8 cards to 4. None of that appears in the model card, the quantization fields, or the SGLang docs. The vendor’s inclusionAI/ling-cookbook repository (head 5d93a6c1, last pushed 2026-10-12T12:49:11Z) holds 162 files, and its 16 mxfp4 recipes all sit under 3.0 paths — ling-3.0-flash (4), ling-3.0-flash-fin (4) and others — with no 3.1 entry at all. That repository is a different place from the SGLang documentation site, where the InclusionAI/Ling-3.1-flash page does carry 3.1 recipes. Treat those three figures as a single unverified source.
Community traction has barely started. A Hacker News submission of the repository on October 15 sits at 1 point with no comments. An earlier comment on the site, dated October 13, used the model as a comparison baseline (“weaker than Ling 3.1”), which says more about it having a seat at the table than about the weights.
Our take
The value here comes from three things at once: an MIT license, one FP8 checkpoint of 573,753,281,152 bytes (about 534.3 GiB), and a day-0 recipe that runs on 8×H200. For teams that already run inference, the third item is the practical one: NEXTN speculative decoding ships inside the container image and the SGLang docs publish acceptance lengths, so expected gains can be estimated before touching production. Conversely, a 534 GiB checkpoint plus an 8-card H200 floor puts this out of reach for anyone self-hosting on consumer or single-workstation hardware; the realistic users are teams with existing clusters and developers calling the model through a managed provider such as OpenRouter.
Two things deserve a discount. First, the 1M context is not available: config.json and the vendor documentation both hold at 262,144. Under the September 30 schedule, the 1M window and the open-source release were both slated for the end of the free period — the weights arrived first and the 1M window has no date attached. Second, keep the three performance numbers apart: Artificial Analysis’s 213.2 tokens/s is a hosted-API measurement, SGLang’s 251.2 tokens/s is a batch-size-1 self-hosted measurement on H200, and Bailing’s long-task results come from the vendor’s own harness with no third-party replication. All three are valid; none belongs in the same comparison table. The weight size works the same way — total_size is the only checkable number, and 534.3 GiB and 573.8 GB are unit conversions of it, not two sources. If the MXFP4 figures are confirmed in official material later, they can be added then — quoting them now would only lend credibility to a single source.
What to watch: whether the weights get an update when the 1M window opens with paid service, and whether the vLLM MTP path matures enough to work without the day-0 image.
How to try it
Start by reading rather than downloading. The Hugging Face repository page exposes the model card, config.json and the install commands without a login, and ModelScope carries a same-named repository (created 2026-10-14T03:00:14Z, License=mit).
If you do run it, follow the model card’s SGLang path: 8×H200 with --tp 8 --ep 8 or 4×GB300 with --tp 4 --ep 4, plus --speculative-algorithm NEXTN. Without that hardware, the managed route is simpler — OpenRouter lists inclusionai/ling-3.1-flash at a 262,144-token context, matching the weights themselves; Bailing’s own service and Ling Studio need an Alipay account to activate.