uni-robot-agent: one harness for robot-use agents, with Codex and Claude Code runtimes

Embodied-AI research code tends to fragment: switch the simulator and the tool interface needs rewriting, switch the agent runtime and it needs rewriting again. uni-robot-agent (291 stars, MIT, Python, created November 2, 2026), open-sourced by Open-RUA, is the harness in the middle — one tool interface separating the agent from the robot environment, with Codex and Claude Code runtimes built in. Version 0.1.1 fixed RoboDojo setup under isolated simulator dependencies and made robot configuration automatic.

Core features

• One interface: shared tool interfaces and feedback across environments, with native APIs handled by adapters, so changing simulators does not touch agent-side code.
• Four operating modes: Direct uses robot tools; Hybrid stacks a compatible action model such as Pi0.5; Code as Policy controls robots with code; Minimal offers a three-tool baseline (move, hold, finish) for controlled comparisons.
• An honest RGB-only setting: the paper’s setting gives only proprioception, calibration and robot descriptions — no simulator depth and no task-verifier feedback — testing agents under restricted information.
• Replaceable seams: agent runtime, model provider, tools, environment and model services all swap through explicit interfaces, so experiments change one variable at a time.
• Broad environment coverage: RoboDojo dual-arm manipulation; RoboUse (tasks from LIBERO, RoboCasa, MetaWorld, RoboSuite, ManiSkill, Tabletop, Fetch and Kitchen); LIBERO-Plus/PRO; RoboTwin 2.0’s ALOHA AgileX dual-arm tasks; RMBench memory-dependent tasks; CALVIN; RoboLab’s DROID tasks; and DexJoCo dexterous-hand work on a Panda arm with a 16-DoF Allegro hand.

Typical use cases

• Reproducing manipulation papers: the RGB-only setting and the three-tool baseline exist for ablations — flip a flag to change the condition.
• Wiring a coding agent to a robot: Codex or Claude Code acts as the brain while the harness translates tool calls into robot actions.
• Comparing agent runtimes: same task, different runtime, action model or environment — configuration only.

Quick start

On a Linux GPU host, create a Python 3.11 virtual environment, pip install uni-robot-agent, then uni-robot-agent setup --integration robodojo. With UNI_ROBOT_AGENT_API_KEY and UNI_ROBOT_AGENT_PROVIDER_URL set, uni-robot-agent run --integration robodojo --task stack_bowls --action-profile direct --no-sam3 --monitor runs a stacking task. --monitor prints a local dashboard URL showing camera streams, tool calls and timing; replay afterwards with uni-robot-agent monitor --run-dir ./runs. Custom strategies go through the Python SDK’s Session.tools.

Relation to existing options

It replaces neither the simulator nor the VLA model: RoboDojo and LIBERO remain environments, Pi0.5 remains an action model, and uni-robot-agent owns the layer of how an agent calls them and how failures come back. It shares its design instinct with harnessrouter, covered here earlier — decouple runtime from tools — except one targets general coding agents and the other targets robot manipulation.

Summary

uni-robot-agent fits teams doing manipulation research, embodied-AI experiments and runtime comparisons. Facts and limits: MIT licensed, created November 2, 2026, so 291 stars is very early; the public package covers simulation only — ARX One and Franka are marked “coming soon” and the project states it does not yet include a supported physical-robot workflow, so real hardware means writing your own integration; the OpenCode and Gemini CLI adapters are experimental, needing separate CLI installs and compatibility validation, with Gemini also requiring the ACP bridge.