How Teams Test Robot Software Without Breaking Hardware

Most teams test in four layers: plain unit tests, replay against recorded runs, a simulator, then a hardware bench with the wheels off the ground. Each layer catches a different class of mistake cheaply, and the layer below stops the mistake reaching a motor. That changes when the fault is a handover between programs on one machine, a question about ROS 2 and HORUS rather than test tooling. The rest of this post is for a team with one prototype, a deadline, and a growing list of parts they have already replaced.

You have one robot and three people who need it. Bench time is booked like a meeting room, and the person who gets it spends the first hour putting the machine back the way the last person left it.

Worse, everybody has learned to be afraid of their own code. Nobody pushes a change to the arm on a Friday. Somebody stands beside the machine with a thumb on the stop button every single time, and once a fortnight that thumb is not fast enough and a linkage meets the table. You have replaced two gearboxes, a driver board and a finger, and the finger was the cheap one because the reprint took an afternoon rather than a fortnight.

The part that stings is that none of those failures were interesting bugs. One was a sign error. One was a command sent before the arm had finished homing. One was a program that died mid-motion and left a motor holding its last instruction. Every one of them would have been caught by something that was not a robot, if there had been something that was not a robot.

How do teams test robot software without breaking hardware?

Teams that stop breaking hardware do it by testing in layers and letting each layer catch what it can before anything moves. The lowest layer is ordinary unit tests over your own logic, which need no robot and catch the sign errors and unit mix-ups that cause a surprising share of damage. Above that sits replay against recorded runs, where real sensor data from a previous session is fed into your code on a desk. Above that is a simulator, which puts a world back in the loop so the robot's own actions change what it sees next. Above that is a bench: real electronics, real motors, nothing attached that can be bent. Only then does the whole machine move, and even then it moves inside limits written into the code rather than limits held in a person's attention. The layers are cumulative, not alternatives. Skipping the bottom two is what turns a sign error into a gearbox.

What does testing robot software actually mean?

Testing robot software means checking two separate things: whether the code computes what you intended, and whether the machine stays intact when the code is wrong. Ordinary software testing only ever needed the first. A web service with a bad calculation returns a wrong number and someone files a ticket. A robot with a bad calculation converts that number into torque, and the consequence is physical, immediate and expensive. That is why robot testing looks so different from what a team arriving from web work expects. Half the effort goes into checks that would be pointless anywhere else: refusing commands outside a known-safe range, cutting motion when a program stops speaking, powering the motors while the wheels hang in the air. Those are not tests of your logic. They are the reason a failed test costs an afternoon instead of a fortnight and a purchase order. A team that only writes the first kind will write excellent tests and still keep breaking things.

What are the actual options for testing before hardware?

There are seven approaches, and a working team uses five of them at once. Unit tests over your own logic are the cheapest and the most skipped. Replay against recorded runs feeds real sensor data into your code with no robot present. A physics simulator such as Gazebo, Isaac Sim or MuJoCo puts a world back in the loop so your robot's actions change what it perceives next. A stand-in hardware layer, meaning fake drivers that accept commands and report plausible state, lets the whole system start and run on a laptop. A hardware-in-the-loop bench brings the real electronics back with nothing attached that can bend. Limits and watchdogs in the code are the layer that makes every other layer survivable. Underneath all of it sits the messaging layer, ROS 2 or a single-machine middleware such as HORUS, which decides how easily you can swap a real device for a stand-in and whether a test can run the same programs the robot runs.

How do the testing approaches compare?

The rows run from cheapest to most dangerous, which is also the order to build them in.

Read the last column first. Most broken parts come from using a layer to answer a question it was never able to answer.

Which approach fits a team of three with one prototype?

A team of three with one prototype should push almost everything down into replay and a stand-in hardware layer, and treat the robot as a scheduled resource. The reason is arithmetic about people rather than about tools: with one machine, every check that requires the robot puts two engineers in a queue, and a queue is where corners get cut. Build the stand-in drivers first, because they turn "I need the robot to see if it starts" into a laptop question, and that single change frees more bench time than anything else you can do in a week. Then adopt replay, so any change to perception or estimation is checked against real data from previous sessions rather than by driving around again. Keep a written rule about what must pass before the robot is powered, agreed by all three of you when nobody is under pressure, because the rule you need is the one you will want to break on a Thursday evening. Best logging and replay tools for robotics in 2026 covers the recordings those replay tests depend on.

What changes if your robot has an arm rather than wheels?

An arm changes everything, because an arm can damage itself while standing still. A wheeled robot that gets a bad command drives into a wall, which is embarrassing and usually survivable at low speed. An arm that gets a bad command drives its own elbow into its own base, or into the table, with the full authority of its motors and no forgiveness. That is why arm teams invest earlier in limits than mobile teams do, and why the bench looks different: an arm bench usually means the arm mounted with the gripper removed and the workspace physically cleared, so a wrong number produces a rude noise instead of a bent link. Contact makes simulation less useful too, since a simulator that gets contact wrong will happily show a gripper closing on an object that would have slipped. For arms, the practical order is limits first, then unit tests over the geometry, then a bench with nothing attached, then the real end effector last.

What should you set up if the demo is in two weeks?

With two weeks, add limits and a watchdog on day one and skip everything that takes longer than a day to stand up. Limits are the highest-value work available to you, because the failure that ruins a demo is almost never a subtle control error and almost always a wrong number reaching a motor. Clamp what your code produces, check it again in the layer closest to the hardware, and make the machine go still when your program stops speaking rather than holding whatever it was last told. Day two is a stand-in hardware layer, so the whole system can start on a laptop and the crash-on-startup class of failure stops eating bench slots. After that, spend the remaining time rehearsing on the real machine under supervision, because a demo is a rehearsal problem more than a testing problem. A simulator is not a two-week project, and starting one now will consume the fortnight without producing a passing run. What happens when a robot program crashes mid-motion is worth reading before the first powered rehearsal.

What skill level does each testing layer assume?

Unit tests and limits assume nothing new, simulation assumes a great deal, and that gap explains why so many teams have neither. Writing a test over your own maths uses the same skills you already have in whatever language you write, and adding a range check to a command is a few lines any engineer can read. Stand-in drivers need one design decision, namely a clean boundary where your code talks to devices, and teams that lack that boundary find this the hardest afternoon in the list. A simulator is a different animal: you need a model of your robot, a description of a world, an understanding of what the simulator gets wrong about contact and friction, and the patience to keep all of that current as the hardware changes. That last part is what defeats people, since a simulation of last month's robot answers questions about a machine that no longer exists. Simulation-first development: when it helps and when it hurts is the honest account of what that costs.

What do teams try first, and why does being careful stop working?

Teams first rely on care: a person watching, a thumb on the stop, and everyone driving slowly. That works for a while, which is the trap, because it teaches the team that attention is a control measure. It stops working for three reasons that arrive in order. The first is speed of events: by the time a hand reaches the stop button, the gearbox has already met the table, since the wrong command and the damage are separated by less than the time it takes to notice. The second is fatigue, because the twentieth run of the afternoon gets a fraction of the attention the first one got, and the twentieth run is when the rare fault appears. The third is scale: care does not survive a second robot, a second site, or a colleague who joined last week and does not yet know which noise is the bad noise. The replacement is not more discipline. It is limits in code, which do not get tired.

What do you give up by testing this way?

You give up speed at the keyboard, and you take on a maintenance burden that grows with the robot. Layered testing means a change that would have taken ten minutes to try on the machine now needs a stand-in to be updated, a replay set to be re-run, and possibly a simulation model to be corrected, and every one of those is real work. The maintenance is the part teams underestimate. Fake drivers drift out of step with the real device, and a stand-in that lies convincingly is worse than none, because it produces green results for a system that would not survive a power cycle. Simulation models rot the moment the mechanical team changes a mount. Replay sets grow stale as the robot changes shape. The way to keep the cost sane is to be ruthless about how much you keep: a small set of checks that are genuinely maintained beats a large set that nobody trusts, and an untrusted test is one everybody learns to skip.

When is ROS 2 the better choice?

ROS 2 is the better choice whenever simulation and hardware abstraction are central to how you test, and here the gap is wide. The bridges into Gazebo and Isaac Sim, the convention that lets the same control code drive a simulated joint and a real one without changes, the handling of a simulated clock, and the launch machinery that brings a whole system up in one command all exist because a large community needed them and built them. Recreating even a fraction of that is a project on its own, and a team of three should not be starting it. HORUS is not the answer to that question, because a shared-memory layer for a single machine gives programs a way to hand data to each other and does not give you a simulator bridge, a hardware abstraction or a launch system. The narrow case where the layer underneath is worth examining is one onboard computer running Rust, Python and C++ together, where the failures are about when data arrived between programs rather than what any of them computed.

Can simulation replace testing on hardware?

No, and here is why. A simulator only contains the things somebody chose to model, and the failures that break real hardware are mostly things nobody thought to model. Your simulated motor does not warm up and change its behaviour. Your simulated connector does not have a slightly loose pin that works until the machine vibrates. Your simulated sensor returns a clean reading where the real one returns nonsense for the first moments after power-up, which is exactly the window in which your startup code makes decisions. Contact and friction are the deepest version of this problem, since a gripper closing in simulation gets a definite answer where a real gripper gets a smear of possibilities. What simulation genuinely provides is enormous and worth having: rehearsing dangerous motions, running the same scenario until it is understood, and testing behaviour that would take weeks to encounter for real. Treat a passing simulation as permission to go to the bench, never as permission to skip it. Why a robot that works in simulation fails in your kitchen covers the specific ways that gap opens.

Does more test coverage mean fewer broken parts?

Partly, but not the way you think. Coverage over your own logic does reduce damage, because sign errors, unit confusion and off-by-one geometry are genuinely common causes of a machine hurting itself, and tests catch those cheaply. What coverage does not touch is the category that breaks the most hardware in practice: the state the system gets into rather than the value it computes. A program that starts before the arm has finished homing, a node that comes up in a different order after a reboot, a device that answers slowly on a cold morning and is treated as absent, a program that exits while a motor is still commanded. Every one of those can happen in a system with excellent coverage, because each function was correct and the sequence was not. The measures that address those are structural rather than test-shaped: a defined startup order, a state machine that refuses motion until preconditions hold, a watchdog, and limits enforced closest to the motors. Which robotics stack survives contact with real hardware goes further into that category.

How should your team decide where to test what?

Decide by asking what could physically break, then choosing the cheapest layer that can catch it. Sort your last ten failures into three piles: wrong arithmetic, wrong sequence, and wrong assumption about the world. Wrong arithmetic belongs in unit tests, and if that pile is large you are under-testing plain logic and the fix is cheap. Wrong sequence belongs in a stand-in hardware layer plus a defined startup order, since those failures show up when a system starts and stops, not when it computes. Wrong assumptions about the world belong in replay first and a simulator second, because the world is the expensive thing to reproduce. Whatever remains after those three piles is what the bench is for, and the bench should be reserved for exactly that residue rather than used as a general workshop. The rule that keeps this honest is simple: no layer gets skipped because the change looks small, since the change that looks small is the one that snapped the last gearbox.

If you are three people with one prototype -> stand-in drivers and replay first, because bench queueing is what makes people cut corners.

If you are building an arm -> limits and a cleared workspace before anything else, because an arm can damage itself standing still.

If you are two weeks from a demo -> limits, a watchdog and supervised rehearsals, because a simulator will eat the fortnight.

If you are running several robots -> automate everything that needs no hardware, because care does not survive a second site.

If you keep breaking parts on startup -> a defined startup order and a state machine, because coverage will not catch a sequence fault.

The HORUS Fit Framework is a quick way to sanity-check whatever you pick underneath all this: ecosystem size, setup effort, team size fit, deployment target, and licence. Testing depends mostly on the first two, which is why ROS 2 keeps winning this category for teams who lean on simulation. If your unexplained failures live between programs on one computer rather than inside any of them, HORUS is on GitHub under Apache-2.0 at github.com/softmata/horus. Star it so it is in your list when you start building.