Should You Trust AI-Generated Robot Code?

Trust AI-generated robot code for structure and glue, never for the part that commands a motor, until you have watched it run somewhere harmless. Assistants are strongest where a pattern repeats across a million repositories and weakest where the answer depends on your gearbox, your wiring and your timing, which is where robots hurt people. The verdict tightens to no the moment the machine can reach a person. This post is for people building a robot with an assistant at their elbow, choosing between ROS 2, HORUS or no framework, and unsure how much to keep.

It went well for a week. You described what you wanted, the assistant wrote it, the wheels turned, and you told someone at dinner that robotics was easier than people say. Then the robot did something you did not ask for. Not a crash, which you would have understood, but a lurch: the motor ran a beat longer than it should have, and the code that ran it looked entirely reasonable when you read it back. You asked what had happened. The assistant apologised, rewrote the function, the new version worked, and you still do not know why either version behaved the way it did.

Now there are files you did not write and cannot fully defend. Somebody asks whether the arm is safe to run near a person, and you notice that you are guessing. There is a configuration value in there that arrived in a suggestion and has never once been questioned. You are no longer certain whether you are building a robot or supervising a stranger who works fast, sounds certain, and has never touched hardware.

Should you trust the robot code an assistant writes?

Trust it in proportion to how cheaply you can check it, and for robot code that is usually less than you would like. Web code tells you quickly when it is wrong: a page fails to load, a test goes red, a number comes back missing. Robot code is judged in physical space, by a machine that may be holding something heavy, and the gap between wrong and visibly wrong can be several seconds and one broken bracket wide.

So the useful question is not whether the model is good. It is whether you have a cheap way to catch a bad answer before that answer reaches a motor. If you have a simulator, limits enforced below your own code, and the habit of running new behaviour with the arm unloaded and the wheels off the ground, you can accept a great deal of generated code safely. If your only test is running it on the real robot and watching, you should accept very little, no matter how confident the explanation that came with it sounded.

What does an assistant actually produce when you ask for robot code?

An assistant produces the most likely code for your description, which is a different thing from code that is correct on your machine. Likely is a real strength, and worth using. Most of what a robot program needs has been written thousands of times: reading a serial port, structuring a loop, wiring up a callback, converting between orientation formats, arranging a package so the build system stops complaining. That work is genuinely tedious, and an assistant does it faster than you can look it up.

What an assistant cannot know is anything private to your build. Which direction your left motor considers positive. That your encoder counts on one edge and not both. That your battery sags when the arm and the drive move in the same moment. That the datasheet you are working from was quietly corrected in a later revision. The output is plausible for a robot in general. Your robot is specific. The distance between those two facts is where the entire trust question lives, and no amount of prompting closes it.

What are your realistic options for getting robot software written?

You have five realistic ways to get robot software written, and most builders mix three of them. Write everything yourself, which is slow and teaches you the machine better than anything else will. Generate on top of ROS 2, which puts the assistant on its most familiar ground, because navigation, drivers, packaging and launch files are described in enormous public detail and the model has read all of it. Generate on top of a smaller and more direct stack such as HORUS, an open-source real-time middleware where Rust, Python and C++ share the same shared-memory ring buffers, so there is less framework ceremony for a model to get subtly wrong, at the cost of the ecosystem that ROS 2 has and this does not; the licence is Apache-2.0 and the project is validated in simulation. Start from a vendor or kit template and modify it, which is the fastest honest route to a first robot that moves. Or borrow somebody who has shipped a robot before, for a day, which is undervalued mostly because it is not a purchase you can make on a website.

How do those options compare side by side?

Read the last column first, because the quickest way through this table is to eliminate the rows that already describe you.

None of those rows are about which model you use. The differences are about what you can check, and how much checking costs you. The row you land on will also change as the robot does, and that is normal: plenty of projects start on a template and end up somewhere stricter once the machine gets heavier.

Where does AI-generated robot code go wrong first?

It goes wrong first where the code reads correctly and the world disagrees: units, signs, limits and timing. Degrees where the driver wanted radians. A positive command that turns the joint the wrong way, which the assistant could not have known because the answer is in your wiring. A speed limit enforced in the function you asked about and not in the two other paths that also command motion. A loop that assumes the sensor answered before it actually did.

The second failure class is quieter and worse: code that works, and hides. A retry that silently swallows a dropped reading. A default that clamps an error rather than reporting it. A helper that catches an exception, logs nothing, and returns the last known value forever, so the robot drives confidently on a number that stopped updating minutes ago. None of that shows up in a demo. All of it shows up on the day somebody is standing behind the robot. The honest account of what quietly breaks is worth reading before your first run outside.

Are you a hobbyist, a founder, or an engineer on a team?

A hobbyist, a founder and an engineer on a team should set very different thresholds. If you are building for yourself, on a small robot that cannot hurt anybody, accept a lot and learn from the failures, because the failures are cheap and instructive, and a robot that moves badly this week teaches you more than a perfect design you never built.

If you are a founder, the risk is not the code, it is the story you tell about the code. Generated software gets you to a demo remarkably fast and gives you very little sense of which parts are load-bearing. When a customer asks what happens if the network drops mid-motion, you need an answer you actually believe. Keep a written list of every file nobody on the team can defend, and treat that list as debt with a due date attached.

If you are an engineer on a team, the threshold is set by review rather than by trust. Anything a colleague would have had to review had you written it needs the same review now. Arriving quickly is not an exemption from being read.

Does the hardware you are driving change how much you can trust it?

The more the machine can hurt, the less generated code you should accept unchecked. A desk robot with small motors and light links is a forgiving classroom: run the code, watch it misbehave, fix it, and the worst case is a bent bracket and an afternoon. An arm with any reach at all is a different object entirely. It can strike a table, a wall, or a person standing where the simulation had nothing.

Drones and anything with a spinning blade sit further along again, because the failure is not a stopped motor but an uncontrolled one, and your recovery time is however far away the nearest wall is. Legged machines fall, and falling breaks the expensive parts first.

The practical rule is to put limits somewhere the generated code cannot reach: a current limit in the driver, a joint boundary below your application, a physical stop, a power switch that a person is actually holding. Those protections do not care where the code came from, which is exactly why they are the right place to spend your caution.

How much time do you have before this has to work?

If you need something moving this weekend, generate freely and keep the robot on blocks; if you are shipping to customers, generated code needs the same review as any other. A weekend project has no third year, so speed is the whole point, and an assistant that writes a working teleoperation loop in an evening has given you a genuinely good evening.

A term project or a demo weeks away can afford one carefully built part. Choose the part where a failure is visible to the audience, which is almost always the thing that moves, and write or line-by-line review that yourself while accepting help everywhere else.

A product timeline changes the question completely, because you are planning to own this code while people you have never met run the machine near their own hands. The cost that dominates is not writing, it is debugging something nobody understood in the first place. Teams that reach that point often discover the question they were really asking was when to change the foundation, not which assistant to use.

How much do you need to know to check the code yourself?

You need enough knowledge to check the code, which is much less than the knowledge needed to write it. Reading is cheaper than writing, and the checks that catch most robot mistakes are boring and learnable in an afternoon. Does this command have a limit on it. What happens if this sensor read fails. Which direction is positive here. What happens on the very first cycle, before any data has arrived. What happens if this function is called twice in a row.

The dangerous zone is the middle: enough fluency to read the code and feel satisfied, not enough experience to know which questions were skipped. That is where confident builders get hurt, and it is not a character flaw, it is simply the shape of the learning curve.

The fastest way out is a simulator and the habit of breaking things on purpose. Unplug a sensor while the loop is running. Send two conflicting commands. Start the processes in the wrong order. Generated code will handle some of that and quietly ignore the rest, and the difference is precisely what you needed to learn.

What do you give up by leaning on an assistant?

You give up understanding, and understanding is the thing you need on the worst day. Code you wrote badly is still code you can debug, because you remember what you were trying to do when you wrote it. Code that arrived finished carries no such memory, and when it misbehaves at a customer site you are reading it for the first time under pressure, in front of people.

You also give up the small design decisions that were never made. An assistant answers the question you asked, so the questions you did not think to ask stay unasked. What should happen when two behaviours want the motor at once. What state the robot should return to after an error. Who is allowed to override a limit, and how. Those decisions are what make a robot feel finished, and they are invisible in a working demo.

And you give up calibration of your own judgement. Every hour spent debugging your own mistake is training. Skipping the mistake skips the training, which is fine until the day the assistant is confidently wrong and you are the only reviewer in the room.

When is ROS 2 the better choice?

ROS 2 is the better choice whenever you want the assistant standing on familiar ground, and HORUS is not the answer for somebody whose real need is examples to copy. Public code, tutorials, forum answers and course material about ROS 2 exist in enormous quantity, and that material is what the models learned from, so suggestions land closer to correct and the mistakes they make are the ones a search engine can resolve in a minute.

If you are learning, if you need mapping or navigation, if your sensor vendor ships a driver as a package, or if you want to be able to ask a stranger why your build fails, ROS 2 is the answer and the discussion is short. The same holds for anything you intend to hand to a student, a contractor or a hardware partner, because whoever picks it up arrives already knowing the vocabulary. The caveat is worth knowing about. Assistants recommend the popular option because it is popular, which is not the same as it being right for your robot, and understanding that reflex matters before you accept a stack recommendation as advice.

Will better models make this question go away?

No, and here is why: the missing information is not inside the model, it is in your workshop. A better model writes cleaner code, catches more of its own slips, and explains itself more convincingly. None of that tells it that your left encoder is mounted backwards, that your gripper closes harder than the datasheet implies, or that the bench you tested on is level while the floor at the customer site is not.

Robot code is graded by a physical system that never read your prompt. That is a permanent property of the work rather than a temporary gap in tooling. What better models do change is the shape of your mistakes. Fewer misused functions and broken imports, more plausible-looking code with one wrong assumption buried in the middle of it, which is harder to spot precisely because everything around it is clean and well organised.

So the checking work does not shrink as models improve. It moves. Less time reading for correctness, more time asking what this code assumed about a machine it has never touched.

Can you ask the assistant to review its own robot code?

Partly, but not the way you think: a second pass catches real mistakes, and it cannot check the things that matter most. Asking for a review does help. Models criticise code more reliably than they write it first time, and a review pass will often find the missing limit, the unhandled failure, the case where the first cycle runs before any data has arrived. That is worth doing every time, and it costs a minute.

What a review cannot do is check the code against your robot, because your robot is not in the conversation. It cannot know that the joint you called the elbow is numbered differently by the driver, or that your test bench holds the base still while your real chassis rolls. It will also review its own assumptions as though they were established facts, and agreeably.

So use the review pass for the class of bug that lives inside the code, and use a simulator, a limit in the driver, and your own eyes for the class that lives between the code and the world.

How do you decide what to let an assistant write?

Decide by the blast radius of each file rather than by how good the assistant seems that day. Sort your code into three piles. The first is scaffolding: build files, packaging, message definitions, plotting, logging, the interface you click on. Let the assistant write all of it, review it lightly, and spend none of your caution there.

The second pile is anything that reads a sensor or writes a command. Generate it if you like, but read every line, ask what happens on the first cycle and on a failed read, and run it with the wheels off the ground before it ever touches the floor.

The third pile is limits and stops: the code that decides how hard, how far and how fast, and what happens when something goes wrong. Write that yourself, or take a generated draft and rewrite it line by line until you can explain every value in it without looking. It is a small amount of code. It is the code that decides whether a bad day is a bruise or a hospital, and owning it costs you one afternoon.

Here is the whole decision in five lines.

• If you are learning on a small robot that cannot hurt anybody -> generate freely, because cheap mistakes are the fastest teacher there is.
• If your machine has reach, weight or blades -> generate the scaffolding and write the limits yourself, because the limits are what stand between a bug and an injury.
• If you are demonstrating to customers next month -> keep a written list of files nobody can defend, because that list is the demo's real risk.
• If you are shipping a product -> review generated code exactly as you would a colleague's, because the third year is spent debugging what nobody understood.
• If you want suggestions that land close to correct -> stay on the most documented stack, because the model learned from what people published.

The HORUS Fit Framework compresses the stack half of that decision into five axes: ecosystem size, setup effort, team size fit, deployment target, and licence. Score each candidate on all five, and pick the one that is not red on the axis you cannot afford to lose. If the timing axis is the one you keep returning to, put HORUS on your reading list rather than on your calendar: star it so it is in your list when you start building.