Weight-space programming edits Qwen3.5-0.8B weights, with scores rising as loops fall
MIT CSAIL reported on Nov. 2 that GPT-6 Astra changed 0.016% of Qwen3.5-0.8B’s weights; MATH-500 rose from 7.9% to 48.1%.
Published entries across all sections carrying the “Benchmark” tag, newest first by publication date on this site.
58 entries
MIT CSAIL reported on Nov. 2 that GPT-6 Astra changed 0.016% of Qwen3.5-0.8B’s weights; MATH-500 rose from 7.9% to 48.1%.
Anthropic released Claude Fable 5.2 and Mythos 5.2 on November 9, 2026. Fable is generally available at $8 per million input tokens; Mythos requires approval.
Shizuku AI’s Shizuku-Duplex listens while speaking in 80 ms ticks. The company reports a 0.57-second median reply, excluding network time.
A look at Vector Graph RAG’s two-stage Jev reranking, what its 86.61% Recall@5 result measures, and how to test it on your corpus.
Per the LobeBench author's Nov 7 test, live DeepSeek V4 Pro jumped from 48% to 64% (32/50), first among Chinese models; unconfirmed, release rumored in a week.
Cloudflare and VoidZero shipped tsrs, a Rust port of the TypeScript compiler built by AI agents in weeks: up to 8x faster per Cloudflare, 5.3x independently.
Discord's CTO confirmed on Nov 4 that its desktop client is being rewritten in Rust, dropping Electron: the new Windows build shows about 290MB RAM vs 1GB+.
Tencent shipped Hunyuan Hy4 on November 4: its first model with vision and computer use, hallucination down to 5.45% per Tencent, open weights promised.
nv-uefi-unlocker boots from a stick and unlocks capped RTX Tensor Cores before the driver loads — a 4060 Laptop gains 65% FP16, no VBIOS flash, no persistence.
Copilot Fusion went GA on October 28: GitHub claims 60 on the AA Coding Agent Index, within 2 points of Claude Code, at roughly half the cost.
Google opened Gemini 4 Carbon to paid API users on Nov. 10. Its $2/$10 per-million-token intro price ends Nov. 30, then rises to $4/$20.
SkeLAM (arXiv 2610.31474) grounds latent actions in robot skeletons: ~10 hours of robot data, no extra large-scale pretraining, 7 of 8 DROID metrics first.
A dermatologist's open test on 200 biopsied lesions: a fine-tuned 2015 Inception v3 scored 62% top-1 vs 42% for GPT-6 Astra and Claude Fable 5.1.
The open-source GPU Unlock module rewrites Ampere speed registers before NVIDIA's driver binds, lifting RTX 30 BF16 GEMM up to +80%. Linux only.
AOMedia reports rapid AV2 uptake: Google demoed Chrome playback, YouTube web streams start as soon as early 2027, and AV2 compresses about 30% better than AV1.
Moonshot AI released Kimi K3.1 Pro on October 23: 1M context, $2/$10 per million tokens, Terminal-Bench 4.0 65.7%, weights due November 4.
StreamEval is an Apache-2.0 benchmark for interactive streaming video: five tasks, 300 cases and 15–300 second sessions, with prompts and code public.
Canva released Craft on October 22: its first in-house image model plans a layout before rendering, tops out at 2K, and ships in Dream Lab and the API at once.
OpenAI moved the Decisions API to GA on October 22: 400 questions per request, unchanged $0.10 per million input tokens, cache writes now billed.
RoboColiseum put four frontier models through one Agent2Act harness on Oct 20: Opus 5.5 leads at 76.28, while manipulation still trails specialist policies.
DuckDB 2.0.0 LTS landed October 21, supported to October 2027. New files use storage format 69, which pre-2.0 versions cannot open.
Recursive released Flint on October 21, a 40B/2.5B Apache-2.0 coding MoE. Its model card claims 74.2 on SWE-bench Verified; no independent rerun is public.
fal launched FLUX 3 Max on October 19: 720p at a promotional $0.0425 per second through November 7 (list $0.17), and an independent board ranks it second.
MiniMax launched M3.1 Flash on October 19: $0.15 per million input tokens, $0.60 output. Artificial Analysis scores it 43, below DeepSeek V4.1 Flash 1017 (45).
Three aurora alpha models flashed in OpenAI's API list on October 18 and were pulled; blind testers say they generate images. Unconfirmed.
Code Arena WebDev puts DeepSeek’s new snapshot #1 among open models (1670 points) at about a tenth of Kimi K3 (Max)’s price, per Arena’s official X post.
Murea Labs released InWorld on Oct 17: a 42 GB 8-step Murea-Fast users can download, while WBench's top row is an unreleased 50-step Murea-Pro.
StartLux-Decision v1.1 27B ranks first of 173 models on the community-run Jev Decision Index at 69.12, first on all three parts; weights are non-commercial.
Manifold AI's R²SI scored 46.58 and 39.35% on RoboDojo's agent board Oct 17, running the same GPT-6 Astra it beat; weights are still private.
DeepSeek’s V4.1 Flash 1017 card lists 11 scores above its prior snapshot; IT之家 reports web, app and API updates, with the same API alias and prices.
Replit CEO Amjad Masad said Oct 15 that it no longer works with Anthropic, citing cost, partnership and performance; an unnamed insider says the tie holds.
Elorian released Glimpse on October 16: a visual model that crops, pulls frames and runs code mid-reasoning, priced at $0.60/$3, all claims from the company.
Linus Torvalds posted a 10-patch kernel series written with Claude Code on October 13, measuring 13.3M to 304M reads/s at 64 threads. Reviewers found a race.
Ant put Ling-3.1-flash weights on Hugging Face on Oct 13 under MIT, ahead of its stated plan; config and docs cap served context at 256K, with 1M unopened.
Meta open-sourced Muse Spark Open on October 15: 816B/42B open-weight MoE, 1M context, $1.25/$4.25 per million tokens on OpenRouter.
NVIDIA open-sourced Nemotron 3.5 Super VL on October 15, 2026: a 120B-total/12B-active multimodal MoE with recipes and datasets, plus a 24GB Bolt edge model.
NVIDIA’s October 13 DLSS 5 update runs twice as fast as September’s model, cuts its parameters to about 118 million, and debuts in NBA 2K27 on October 16.
Epic's launcher Preview, rebuilt by a small team with AI, cold-starts in 5.47s vs Steam's 10.07s in Epic-commissioned tests; Fortnite follows on EAC Linux v2.
MIT, ECMWF and Microsoft published probabilistic bias correction in Nature on Oct 14: 2-6-week forecast skill doubles, gains skew tropical, code open source.
Gemini 4 Argon went GA on October 13: paid API and Gemini app for AI Pro/Ultra; intro pricing of $2/$10 per million tokens doubles after November 30.
Z.ai shipped GLM-Legal, a legal agent model built on GLM-5.3-Flash: 19.2% all-pass on Harvey LAB, API by application, 7th of 79 on vals.ai.
Meta's first frontier models landed Oct 12: Muse Wonder at $4/$20 per million tokens, Wonder Full for Muse Code subscribers; independent evals scored 52/56.
Yann LeCun said in Seoul on Oct 12 that AI's next breakthrough will come from world models, not bigger LLMs; his H-JEPA paper lifts AntMaze from 18% to 73%.
Agent leaderboard data through Oct 8 puts Claude Opus 5.5 (Max) first at 15.53% and Opus 5.5 (High) second; DeepSeek, Kimi and Tencent models rank 17th–19th.
Simular's open-source agent that works desktop and web apps from screenshots, clicking and typing with no per-app APIs. README reports 72.6% on OSWorld.
Arena, the company behind LMArena, raised a $200M Series B at a $3.1B valuation — nearly double January — and launched an alignment leaderboard for behavior.
Tavus unveiled Griffin on Oct 1, a full-duplex video-to-video model; 26 of 54 testers took it for a person after a one-minute call. Preview only.
$0.10/$0.50 per million tokens, about 75% cheaper than 4.5, with OSWorld at 72.4% and a first effort dial.
A 1T-parameter (49B active) MoE trained on 3,800 Blackwells in Europe; open weights this month and a €3B Series D.
OpenAI and Amazon shipped Jev-style decision models within two weeks; TypeSafe reportedly talks a raise past $10B, three weeks after its $40M seed.
Google announced Gemini 4 Argon on September 30: 77.9% on DeepSWE above GPT-6 Astra, a 1M-token output limit, phased release, no API pricing yet.
OpenAI shipped GPT-6.1 Sol at DevDay: near-Astra performance at one-fifth of Astra’s prices ($2/$10 per million tokens), live in ChatGPT Work and Codex.
Anthropic released Sonnet 5.5 on September 28: $2/$10 per million tokens (half of Opus 5.5), 30% faster, and better agentic coding than Opus 5.5 in-house.
Open-source SparkDiffusion pushes 720P-14B video generation to 265x on one RTX 5090 via sparse attention, distillation and FP8; VBench-2.0 drops 0.5 points.
Stealth model Pixel Canary went free on Vercel AI Gateway on September 25, tying GPT 6 Astra on Next.js coding evals — at nearly 17 minutes per task.
Shanghai Innovation Institute released the Nex-N2.5 family on September 9, 2026 — 35B, 397B and 1.6T models, all under Apache-2.0. On the official table the 1.6T Max tops BrowseComp at 92.6, ahead of Claude Opus 5 and GPT-5.6 Sol, and outscores GLM-5.3 and DeepSeek-V4-Pro on automation and tool-use benchmarks.
Anthropic released Opus 5.5, the first Claude 5.5 model, on September 23. The company says it matches Fable 5.1 on most tasks and costs 40% less per typical task than Opus 5, at $4 input and $20 output per million tokens.
Inferact, founded by the original vLLM team, wrote a megakernel inference kernel for Google TPUs. Paired with DeepSeek’s DSpark speculative decoding, 16 TPU v7 chips served Kimi K3 at 709 tokens per second versus 452 on GB200 under the same setup, QbitAI reports. The code is open source.