Agentic Visuomotor Policy Learning via Real-to-Sim-to-Real

A coding agent turns one real demonstration per task into a simulated scene, randomized demonstrations at scale and a visuomotor policy. The policy is trained only in simulation and then run on real hardware.

Mug hanging
Tape in box
Cup nesting
Cable routing

Real-time policy rollouts on the real robot.

From one demonstration to a deployed policy

Overview: single real demonstrations of four tasks, a grid of randomized simulated scenes, and real policy rollouts near task completion, connected by the agent commands run-real2sim and run-sim2real
Left: one real demonstration per task. Middle: the agent reconstructs each scene in simulation (/run-real2sim) and generates randomized demonstrations at scale. Right: the visuomotor policy it trains (/run-sim2real) running on the real robot, near the end of each task.
Method: the coding agent runs real-to-sim reconstruction for rigid, articulated, deformable and erasable-ink scenes, then writes a task expert, designs domain randomization, generates demonstrations at scale and trains and evaluates a policy
The coding agent runs two workflows. Real-to-sim reconstructs the scene from the recording, reproduces the demonstration in simulation, and reports which sim-to-real gaps were measured and which stay uncertain. Sim-to-real writes a task expert and revises it from test rollouts, designs domain randomization informed by the gap report, generates demonstrations at scale, and trains and evaluates the policy in closed loop. A human records the demonstration, sets up each real trial and starts every robot run.

Real-to-sim beyond rigid objects

0.0 s
Rigid: mug and stand, the scene behind the deployed mug policy. The simulation runs the agent’s task expert in the reconstructed scene. Both clips play in real time from the same camera and start together.

Randomized demonstrations at scale

The agent-written task expert collects demonstrations under randomized lighting, materials, cameras, image effects, object layouts, physics, controller dynamics and external pushes. Pick a kind of randomization to see training episodes that have the most of it, with the values sampled for each one.

Training-set videos as the policy sees them: one frame per supervised sample, so motion is not real time. Values are the episode's recorded draws; scales are relative to the reconstructed scene.

Closed-loop evaluation in sim and real

Sim The policy in closed loop under broad randomization.
Real The same checkpoint on the real robot.

Results

TaskSim nominalSim mildSim broadReal robot
Mug hanging100%1/1100%20/20100%20/2088%22/25Failures: missed pickup, mug not released, and protective stops
Tape in box100%1/145%9/2075%15/2072%18/25Failures: tape placed inside but lid left open
Cup nesting100%1/195%19/2090%18/2080%20/25Failures: incomplete upright nesting or toppled stacks
Whiteboard wiping100%1/175%15/2055%11/2052%13/25Failures: missed grasps or failed release
Cable routing100%1/175%15/2050%10/2076%19/25Failures: missed grasps or incorrect placement

Real rollouts

Discussion

Our experiments highlight both the capabilities and the remaining challenges of agentic visuomotor policy learning. Below, we discuss what worked, what remained difficult, the limitations of our evaluation, and directions for future work.

What worked

In our experiments with the GPT-6 Astra series, agents showed strong capabilities in code-based 3D modeling, carrying out zero-shot real-to-sim reconstruction with relatively little additional scaffolding. Agents could even reconstruct scenes with challenging physics models without being supplied with template code. These spatial reasoning capabilities also appeared to extend to robot control: agents could develop motion plans in relatively few iterations, drawing on the close connection between action planning and reasoning about 3D geometry.

What remained difficult

One interesting observation from our experiments is that, despite their strong capabilities in real-to-sim construction and action planning, agents were less reliable at applying best practices for generating training data and training policies. Without explicit guidance from the harness, they sometimes omitted important sources of variation from domain randomization or chose unsuitable training hyperparameters. For example, they could apply too little visual randomization, choose an action chunk that was too short, or set the learning rate too low.

We found it useful to provide default values in the harness and explicitly explain when and why the agent should override them. These defaults give the agent a starting point, while guidance on adapting them helps it make informed choices for each task.

Detailed harness instructions do not fully solve this problem. They can constrain the agent’s creativity while still leaving gaps in its decisions. For example, an agent may generate an expert plan that satisfies the stated requirements but is not robust, such as grasping an object near an edge rather than choosing a more stable grasp. This leaves a tradeoff between providing enough guidance for reliable behavior and preserving the agent’s ability to explore better solutions.

Limitations

Our evaluation does not sufficiently cover contact-rich, high-precision, or dynamic tasks, or tasks that require open-world generalization. We also only evaluate a specific combination of ACT policies and GPT-6 Astra, though we expect the harness to generalize to other agents and policy architectures.

Furthermore, these experiments do not use large-scale pretrained vision-language-action models, such as π₀.₅, or world action models. We also do not use feedback from real-world deployment to initiate additional real-to-sim-to-real rounds and refine the policies. Incorporating these models and iterative deployment feedback would introduce new opportunities as well as new challenges.

Next steps

Recent growth in agent capabilities suggests that even complex domains can become increasingly tractable through data, computation, and training. From this perspective, the difficulties we observed in generating training data and launching policy training motivate further work on incorporating robotics expertise into agents through post-training and harness design. The goal is to equip agents with the domain knowledge and judgment needed to make decisions at the level of a robotics expert.

Future research could also study the tradeoff between providing enough guidance for reliable behavior and preserving the agent’s ability to explore better solutions. Understanding this balance could inform more effective harness designs. Another direction is to learn from real-world deployment feedback, using observed successes and failures to guide new real-to-sim-to-real rounds. This could support continual self-improvement as agents refine their simulations, data-generation strategies, and policies through repeated deployment. Finally, extending the approach to contact-rich, high-precision, and dynamic tasks, while incorporating useful priors from pretraining, would broaden its scope and test its capabilities under more demanding conditions.

Paper

First page of the paper Agentic Visuomotor Policy Learning via Real-to-Sim-to-RealCoRL 2026 late-breaking results, extended abstract