Research#Multi-agent#Benchmark
Manifold AI's R²SI tops RoboDojo's agent board, running the same GPT-6 Astra it beats
Manifold AI's R²SI scored 46.58 and 39.35% on RoboDojo's agent board Oct 17, running the same GPT-6 Astra it beat; weights are still private.
R²SI — Recursive Robot Self-Improvement — is a two-layer robot agent from Manifold AI. An inner Robot Agent executes; an outer Recursion Agent reviews the execution records, rewrites the procedural skills, and where a task uses a reactive model, retrains that model too. On October 17, 2026, RoboDojo’s simulation leaderboard logged WorldScape R²SI at 46.58 points and 39.35% in-budget success, first on its agent track. The VLM inside the inner agent is GPT-6 Astra — the same model as the 28.97 / 22.48% entry further down that board.
Key points
- WorldScape R²SI scores 46.58 on RoboDojo’s agent tab with a 39.35% in-budget success rate — 17.61 points above GPT-6 Astra’s 28.97 / 22.48% in the reference harness, using the same VLM.
- The report splits the 54 task variants across two execution paths: 26 run through the inner Robot Agent and its skills, 25 go to Simate-beta, and 3 go to Manifold’s own WSP.
- The main sim table’s own title is “RoboDojo Sim Leaderboard (w/o Agent)”; agent entries sit in a separate tab, and Manifold’s package marks itself
eligibility_status: "unverified"with no weights attached.
Background
RoboDojo is a bimanual manipulation benchmark: 42 tasks in five ability groups plus 12 randomized-layout variants, 54 in all. Each task carries an official native step limit — 200 for align_blocks, 700 for swap_blocks — and the board ranks in-budget progress score and success rate. AI MMLab Club, a non-profit foundation, runs it, with hidden layouts as a consistency check.
The top of that board turned over repeatedly this autumn: GPT-6 Astra at 28.97 / 22.48% on September 16, VPP2 at 39.26 on October 8, Einsia.AI’s EmbodiedEvo at 45.08 on October 14, and R²SI at 46.58 on October 17. R²SI leads the agent tab by 1.5 points over an entry that arrived three days earlier.
GPT-6 Astra’s own result has a separate paper behind it: “An Unexpected Robot Policy,” posted September 21, ran 2,100 trials across all 42 tasks and measured 22.48% average success — though that leaderboard entry was also contributed by the RoboDojo Team, so it is not an independent third-party check. The authors found the profile polarized — good on semantic tasks, poor on precision and bimanual coordination — which is where a different harness might help.
The two layers
The inner Robot Agent sees only head and wrist RGB, public proprioception, the task description and episode memory, and acts through the skills loaded for that task. Calling the reactive model is itself one of those skills: the model executes a single action chunk and hands control back, and the agent decides the next step from fresh observations.
The outer Recursion Agent reviews records across rounds, rewrites a short SKILL.md telling the robot how to approach, verify and recover on one task, and sends it to a test gate. The gate requires that the score not drop and that every validation episode stay inside the step limit; only then is the change released and versioned. For reactive-model tasks the outer agent also turns checked segments into training data and updates that model. The release the report describes carries 58 task skills over 24 variants, 5 General Skills and 6 MCP tool packages.
Where the variants run
Section 5.1 states the split plainly: 26 variants run through the Robot Agent and its skills, and the other 28 go to a reactive model — 25 to Simate-beta, 3 to Manifold’s WSP. R²SI beats Simate-beta by 12.6 points while routing nearly half the benchmark to it.
That is not cheating; assigning each task to the executor that suits it is part of the harness, and the report lists path selection as a source of gain. Handing put_bottles_into_dustbin to Simate-beta lifts that task from 35.3 to 98.8 and the Precision group from 12.65 to 23.51. It does change how to read 46.58: the number belongs to two executors, a routing table and a skill library, not to one VLM with better prompting.
WSP is Manifold’s own world-action model, previously released as WorldScape Policy 2.0 and 2.5 with in-context learning from a single demonstration as its pitch. Set beside Yann LeCun’s bet on H-JEPA world models, the difference is purpose: there the world model carries prediction and planning, here WSP is an action generator the agent calls as a tool.
How the loop improves
align_blocks is the report’s worked example: push a thin ruler until it lines up three blocks, then return home, inside 200 native steps. The starting skill follows a generic grasp-and-lift routine and finishes none of eight layouts within the limit. The outer loop writes four candidates. v1 pushes with an open gripper: 8/8 complete, only 2/8 inside budget, mean 215.1 steps. v2 sends position and orientation in one call: 7/8. v3 lifts clear before rotating the wrist on retreat: 6/8. v4 lowers the contact height when a short push fails to move the ruler: 8/8 in budget, mean 140.4 steps. The gate rejects the first three and auto-promotes v4; tool calls fall from 17.0 to 12.3.
The push_T numbers need separating. The project page and 机器之心 both give 723 → 596 → 389 steps against a 600-step budget; the project page adds only that the frozen skill then succeeded on 3 of 3 random layouts, while 机器之心 supplies 222, 312 and 256 steps. The report PDF contains no push_T figures at all — its case study is align_blocks — so those three numbers carry a single-source label.
The ablation is Table 5: swap in the reference harness and the macro score on 25 xhigh variants drops from 56.3 to 43.4; drop the outer revision and align_blocks falls from 8/8 to 0/8; drop the skill layer and mean native steps on stack_blocks rise from 215.5 to 370.5. On cost, a median xhigh episode takes 612 native steps, 3.8M tokens and 15.6 minutes, with 97.4% of those tokens served from the prefix cache.
How to read the board
The same 46.58 has three readings. The main sim table’s title constant is “RoboDojo Sim Leaderboard (w/o Agent)”, and agent entries are filtered into a second table titled “RoboDojo Sim Agent Leaderboard”. Mixed into the main table, agent rows are faded, unranked and captioned “Agent models are shown faded and unranked”. The home page’s Top 10 sorts every entry by score, and R²SI sits on top.
So “first overall” holds for the home-page mix and for the agent tab. The first entry on the main table, agent rows excluded, is RobotEra’s VPP2 at 39.26 / 32.26%. 机器之心’s line that WorldScape R²SI ranks first overall needs that qualifier attached.
| Entry | Type | Score | Success |
|---|---|---|---|
| WorldScape-R²SI | agent-wam | 46.58 | 39.35% |
| EmbodiedEvo | agent | 45.08 | 38.95% |
| VPP2 | wam | 39.26 | 32.26% |
| PhysicalRSI | agent-vla | 36.27 | 31.38% |
| GPT-6 Astra | agent | 28.97 | 22.48% |
Figures from the RoboDojo leaderboard as of 2026-10-17. A baseline like GROOT-N1.7 (2.85 score, 1.31% SR) is a weak comparison; the informative figure is the 17.61-point gap between one VLM under two harnesses.
RoboDojo’s protocol adds that results submitted without the evaluated checkpoint, implementation and reproduction instructions “are reported separately and are not considered verified leaderboard entries”. That clause lands squarely on R²SI’s current state.
What others say
Manifold’s submission is blunt about it. The README says “This is a draft integration, not a verified leaderboard release” and lists what remains pending: real-checkpoint debug evaluation, CUDA motion planning, simulator success, public checkpoint access, an eval-only agreement and the report URL. MANIFEST.json sets official_single_entry_eligible to false; each WSP checkpoint is about 25.27 GB, the weight bytes are not bundled, and download URLs remain pending. Upstream PR #162 in XPolicyLab has been open since October 11 with 8 commits and zero human comments.
No third party has reproduced or critiqued R²SI itself yet. Jay Chia’s Topic Queue post on August 20 counted the 34 policies then listed and found 32 in single-digit success, with the top three submitted by the labs themselves. He noted the benchmark is deliberately hard and flagged that the leader’s weights had zero downloads. Those numbers predate the October wave, so the piece works as a methodological warning, not a verdict on R²SI.
On the English side I searched Google News for three query sets (WorldScape R2SI, Manifold AI RoboDojo, R²SI robot) and got zero items each time. In Chinese there is one 机器之心 article, with Sohu and Phoenix running syndicated copies.
Our take
For anyone building embodied systems, the useful part of R²SI is not 46.58; it is that improvement lives in a text layer anyone can read. A skill is a SKILL.md file, the gate is a set of reproducible conditions, and releases are versioned and revertible — far cheaper to port than retraining a VLA. To try the idea on your own tasks, skip the outer agent at first: write the failure causes down as checks, and put a no-score-drop gate in front of every edit.
The second transferable finding is that executor choice should be explicit. R²SI routes nearly half its variants to Simate-beta, which says “which executor for which task” is a variable worth tuning on its own; one strong reactive model plus an agent that picks its work is easier to keep inside a step budget than one model forced to cover everything.
Three discounts apply. The weights are private, the package calls itself unverified, and the PR is still open, so what is confirmed is that the score is listed — not that anyone can reproduce it from the documentation. At xhigh effort a median episode costs 15.6 minutes and 3.8M tokens, and with 97.4% of tokens hitting the prefix cache the real bill depends heavily on cache behaviour. The three random-layout step counts for push_T rest on one Chinese source and are absent from the report.
For contrast on what a leaderboard claim can mean, look at Arena’s agent leaderboard: that one uses human sessions and confidence intervals, this one uses fixed tasks and official step limits, and their “first place” are not the same kind of first.