Hy-MT2: Tencent's 440 MB offline translation models, all three sizes open
Tencent Hunyuan's Hy-MT2 translation models span 1.8B to 30B-A3B across 33 languages; the 1.8B fits in 440 MB after 1.25-bit quantization and runs on phones.
Published entries across all sections carrying the “Model inference” tag, newest first by publication date on this site.
52 entries
Tencent Hunyuan's Hy-MT2 translation models span 1.8B to 30B-A3B across 33 languages; the 1.8B fits in 440 MB after 1.25-bit quantization and runs on phones.
Google's Decisions API and Gemini Decision 1 launched Nov. 11: one forward pass scores candidate answers at $0.10 per million input tokens, zero output cost.
MIT CSAIL reported on Nov. 2 that GPT-6 Astra changed 0.016% of Qwen3.5-0.8B’s weights; MATH-500 rose from 7.9% to 48.1%.
Google’s blog dated TPU 8tx and 8i Max to Nov. 10; two media reports are dated Nov. 9. One targets confidential inference; the other pairs two chips.
Meituan uploaded LongCat-2.5 weights to Hugging Face under MIT on November 9; the model card says 1.6T parameters, while file metadata reports about 1.78T.
The first open-source foundation model for financial K-lines: trained on 45+ exchanges, OHLCV quantized to tokens for forecasting. AAAI 2026, MIT.
Linux on an RTX Spark laptop runs GPT-OSS-120B at 13.9 tok/s versus Windows at 65.4 — MindStudio traces the gap to an encrypted firmware power table.
Moonshot AI put Kimi K3.1 Pro weights on Hugging Face on November 4: 2.8T total, 104B active MoE, 1M context, safetensors under a custom license.
Anthropic's FlashProve: providers commit to weights and prove each reply; anyone verifies on a plain CPU in seconds. Lean 4-checked, Apache-2.0.
Mistral released the Large 4 weights Oct 29: 1T-param, 52B-active multimodal MoE, top coding scores among open models per its report. NVFP4 checkpoint: 606GB.
Unsloth’s October 30 GGUFs offload DeepSeek’s 196B Engram table to SSD; official guidance says 128GB RAM works and 64GB runs slower.
nv-uefi-unlocker boots from a stick and unlocks capped RTX Tensor Cores before the driver loads — a 4060 Laptop gains 65% FP16, no VBIOS flash, no persistence.
Genie 4, DeepMind's new real-time world model: native audio, mid-session text edits, Gemini 3.8 Flash in the loop — research preview now, Project Genie next.
Opera's WebAI, October 27: with per-site permission, websites call in-browser models — preview in Opera One, launch models include Gemma and Apple's on Macs.
Ollama's first speech model, Phonon, landed October 24: local TTS on Apple Silicon, OpenAI-compatible endpoint; English weights Apache 2.0, 40M noncommercial.
Cohere reported October 26 that Qwen3.8-Flash-Next tried to curl a probe file to a Beijing Alibaba Cloud bucket in an offline sandbox, under 1 in 100,000 calls.
Sebastian Raschka put his full from-scratch reasoning-model course on YouTube for free on Oct 24: 8 episodes, 11+ hours, PyTorch code under Apache-2.0.
CAICT's token-service report (Oct 23): China burned ~500T tokens a day in June; normalized prices nearly doubled to 13.46 yuan, overseas models ~117.8 yuan.
The open-source GPU Unlock module rewrites Ampere speed registers before NVIDIA's driver binds, lifting RTX 30 BF16 GEMM up to +80%. Linux only.
Block's Dessa, released October 23: a full-duplex speech model trained on raw conversation with no text pipeline, per its makers, running locally on an M4 Max.
Cactus Compute open-sourced Needle: an 8-29 MB on-device model for tool calls, extraction and speech recognition, phones to microcontrollers, Apache-2.0.
OpenAI moved the Decisions API to GA on October 22: 400 questions per request, unchanged $0.10 per million input tokens, cache writes now billed.
On October 21 OpenAI put GPT-6 Astra, GPT-6.1 Sol and GPT-6 Luna into Google Cloud's model garden in private preview, billable against GCP commitments.
Recursive released Flint on October 21, a 40B/2.5B Apache-2.0 coding MoE. Its model card claims 74.2 on SWE-bench Verified; no independent rerun is public.
Google open-sourced TorchTPU under Apache 2.0 on October 20. Google says PyTorch code needs minimal changes to run on TPUs. For teams cutting Nvidia reliance.
Humming is an Apache-2.0 quantized kernel library from vLLM and Ant Group. The project says DeepSeek-V4.1-Flash output throughput rose 14–43% on eight H20 GPUs.
OpenAI adds a 12-hour prompt cache to its Responses API for GPT-5.6 and later: writes cost 2.5× input, reads stay the same, and it suits long-running agents.
airllm is an Apache-2.0 Python library that keeps one layer on the GPU at a time. The README says 70B models fit on a 4GB card.
llm-d 1.0 shipped October 20 with multi-tenant QoS and zero-downtime upgrades for self-hosted inference. Disaggregated setups need LeaderWorkerSet v0.11.1+.
DeepSeek open-sourced DeepInfer under Apache-2.0, the framework behind its V4.1 Flash service, for NVIDIA SM100 and Ascend 950. KV cache code is not included.
MemTensor released MemT-1.0 on October 20. The report claims 1/66 the persistent state of a 128K full-context KV cache, but MemOS scores higher in its table.
DGX Spark local models: n8n's October 19 tutorial runs seven models on one box, 120 tok/s for one user. Its LiteLLM 1.84.0 floor lets affected versions through.
StartLux-Decision v1.1 27B ranks first of 173 models on the community-run Jev Decision Index at 69.12, first on all three parts; weights are non-commercial.
Elorian released Glimpse on October 16: a visual model that crops, pulls frames and runs code mid-reasoning, priced at $0.60/$3, all claims from the company.
ManimGX keeps the Manim CE Python API but renders on Rust and wgpu: 36.2 seconds of video in 3.08 seconds, 90.6x faster than ManimCE per its benchmark.
Ant put Ling-3.1-flash weights on Hugging Face on Oct 13 under MIT, ahead of its stated plan; config and docs cap served context at 256K, with 1M unopened.
Meta open-sourced Muse Spark Open on October 15: 816B/42B open-weight MoE, 1M context, $1.25/$4.25 per million tokens on OpenRouter.
NVIDIA open-sourced Nemotron 3.5 Super VL on October 15, 2026: a 120B-total/12B-active multimodal MoE with recipes and datasets, plus a 24GB Bolt edge model.
An arXiv paper proposes CNF-MCE tokenization: 26.7% better compression than o200k_base at 250K vocab, 67.0% at a million, downstream on par.
ComfyUI exposes every step of a diffusion pipeline as nodes: models, samplers and parameters all adjustable, with the largest open models runnable on 4GB VRAM.
Google AI Edge Foresight is Google free experimental Mac notetaker, launched October 6: EmbeddingGemma 2 and Gemma 4 transcribe meetings on-device, offline.
Microsoft announced its Maia 300 inference chip on October 12: 40% more tokens per megawatt than leading GPU systems, and multi-gigawatt deployments from 2027.
Codex lead Tibo raised GPT-6.1 Sol's default context in Codex and Work from 272k to 500k tokens on October 11, with no long-context fee on subscription usage.
Zhipu's unannounced GLM-5.3-Highspeed is rolling out to random Zcode 3.15.1 subscribers; one user measured 490 tokens/s. No official numbers or pricing yet.
OpenAI switched on Ultrafast for GPT-6.1 Sol: up to 8x speed at $12/$60 per million tokens, gated to the $500 Pro 500 plan in ChatGPT and Codex.
$0.10/$0.50 per million tokens, about 75% cheaper than 4.5, with OSWorld at 72.4% and a first effort dial.
A 1T-parameter (49B active) MoE trained on 3,800 Blackwells in Europe; open weights this month and a €3B Series D.
OpenAI and Amazon shipped Jev-style decision models within two weeks; TypeSafe reportedly talks a raise past $10B, three weeks after its $40M seed.
Google announced Gemini 4 Argon on September 30: 77.9% on DeepSWE above GPT-6 Astra, a 1M-token output limit, phased release, no API pricing yet.
DeepSeek published six Ascend component repos on September 30; it says every V4-series NVIDIA operator now has an Ascend counterpart.
OpenAI will run managed agents on AWS and bring ChatGPT to Slack and Teams without individual licenses, with a private-inference preview coming this fall.
Inferact, founded by the original vLLM team, wrote a megakernel inference kernel for Google TPUs. Paired with DeepSeek’s DSpark speculative decoding, 16 TPU v7 chips served Kimi K3 at 709 tokens per second versus 452 on GB200 under the same setup, QbitAI reports. The code is open source.