56% to 95% mean success.
Hand-engineered planner: 47%.
That’s the gap coding agents just opened on generalized task-and-motion planning, and it isn’t subtle.
The source is “Coding Agents for Generalized Task and Motion Planning Problems” (arXiv:2609.30233), written by Matteo Merler, Bowen Li, Josh Roy, Yichao Liang, Qianwei Wang, Yixuan Huang. And Tom Silver, filed under Robotics and Artificial Intelligence.
So can you hand robot planning to a coding agent? On this evidence, yes. Better than the bespoke tools, actually. Three agent configurations — Claude Code running Opus 5, Codex in its GPT-5.6 Sol and GPT-6 Astra configurations. Each got a task description, simulator access, and a fixed synthesis budget. They wrote their own planning programs. Got evaluated across 28 simulated environments built on KinDER and PDDLStream. Beat the hand-engineered planner on the 16 environments where one was available.
And in the object-count comparison they burned an order of magnitude less computation per instance on average.
How the coding agents were tested
Setup’s blunt. Task description in, simulator access granted. Between those two points, nothing. The agent chooses how to interact with the environment while it develops its program. No hand-holding.
It explores. Writes code. Tests. Revises. All inside that fixed synthesis budget, which caps how much program development it can burn before it has to ship something.
Here’s the evaluation surface in numbers:
– 28 simulated environments, sourced from KinDER and PDDLStream
– Object counts pushed beyond what the original benchmark evaluations used
– 3 configurations spanning products you can buy today — Claude Code with Opus 5, Codex as GPT-5.6 Sol and GPT-6 Astra. Not lab-only models stitched together for a paper
– 16 environments where a hand-engineered planner was available for comparison
– 47% mean success for that planner, against 56% to 95% for the agents
The products-you-can-buy part matters more than any single success rate.
Capability that lives in tools your team can already provision isn’t a research promise.
And the whole exercise aims at generalization.
The synthesized programs are meant to generalize across problem instances. Develop one program, it pays off on every later instance instead of evaporating after a single solve.
Robot planning is the brutal test for coding agents
TAMP. Task-and-motion planning. Is hard because discrete decisions are coupled to geometric, kinematic. And dynamic constraints. “Pick up block A, then block B” is the easy half. Whether an arm can physically reach A without colliding with B, whether the grasp holds under load, whether the whole sequence survives contact with continuous physics. That’s the hard half.
The paper’s framing is nastier, though. These difficulties persist even with full observability and object-centric states. Which kills the usual excuse that more sensors or better perception will save you.
Two more claims from the research: pure textual reasoning is insufficient for these problems. And existing generalized-TAMP methods require substantial TAMP-specific engineering. Underline that second one twice. The field’s answer to hard planning has always been to engineer a planner per domain, then engineer another when the domain shifts.
Related work on LLM-generated code for TAMP.
The Code-as-Symbolic-Planner line of research.
Makes the same argument from the other side: language models got good enough at code generation that the symbolic computation can be synthesized instead of hand-built.
That’s the story here.
The agents didn’t get smarter about physics. They got smarter about writing the program that reasons about physics.
Coding agents beat hand-engineered planners: the numbers that matter
– 56% to 95% mean success across the three agent configurations
– 47% for the hand-engineered planner
– Also beats one-shot generation methods and an LLM-based generalized-planning baseline on the same mean-success comparison
– As object counts increase beyond the original benchmark evaluations, the agents’ programs maintain higher success than the planner
– An order of magnitude less computation per instance on average in that object-count comparison
Beating one-shot generation is the result I care about most. One-shot is still how most people demo agents: you paste the task, read the answer, shrug at the failure rate.
Put the last two bullets together and you get the shape of the win. The expensive part is program development inside the budget, and it happens once. Every new instance after that reuses the program cheaply, while the planner grinds through fresh search every time.
Amortization, not magic. It’s too the strongest argument I’ve run into for agents that write code over agents that chat.
What coding agents writing code means for your business workflows
I build agent automations for small businesses. Nearly every failure I clean up traces to one mistake: the agent is paid to answer, not to build. The TAMP result is a robot paper. The pattern transfers anyway. Invoicing flows, intake triage, data cleanup jobs.
Three pieces transfer:
– Give the agent an environment to poke at, not just a prompt. Choosing how to interact while developing a program is where correctness comes from.
– Cap it with a fixed budget. Unlimited revision cycles hide the fact that nothing reusable got produced.
– Demand a program as the output. An artifact you run again tomorrow for free, not a text answer you pay for again tomorrow.
Action item’s concrete. Pick one workflow you currently run as a chat-style agent. Rebuild it as program synthesis with environment access and a hard budget. Measure success before and after. If coding agents can beat 47% hand-engineered planning with 56% to 95% on robot benchmarks, per the arXiv robotics listing, your repetitive business workflows have headroom too. Read the paper first, then go rewrite one pipeline this week.
FAQ
What is generalized task and motion planning?
TAMP couples discrete decisions to geometric, kinematic, and dynamic constraints.
Deciding a sequence of actions while proving each action is physically executable. The generalized version wants a single program that works across problem instances, not one bespoke solve per instance.
Can coding agents replace hand-engineered planners?
On these benchmarks, yes. The agents beat the hand-engineered planner on the 16 environments where one was available, posting 56% to 95% mean success against 47%, while using an order of magnitude less computation per instance on average in the object-count comparison.
What success rate do Claude Code and Codex achieve on TAMP?
Mean success rates of 56% to 95%, across three configurations: Claude Code running Opus 5.
And Codex in its GPT-5.6 Sol and GPT-6 Astra configurations.
How were the coding agents evaluated?
Across 28 simulated environments built on KinDER and PDDLStream, with object counts pushed beyond the original benchmark evaluations.
And each agent capped by a fixed synthesis budget for program development.
Sources
– “Coding Agents for Generalized Task and Motion Planning Problems” — arXiv robotics listing
– Code-as-Symbolic-Planner
