Simulation-First Development: When It Helps and When It Hurts

Simulation-first development is the right default for a robotics team's first months, and hardware-first becomes right the moment the simulator stops surprising you. Simulation buys cheap failures and parallel work, but pays only while the code in the simulator is the code that ships, on ROS 2 or HORUS. That flips once the task depends on contact, because the simulator starts lying and hardware becomes the only honest answer. The rest of this post is for teams deciding how much of the next quarter to spend in a simulator and when to stop.

Your robot works in the simulator. It has worked there for weeks, somebody made a video, and everyone was pleased. Then the real machine arrived, and on the first run the arm reached for the object, closed on nothing, and kept going as if it were holding something.

Now the work has split in two. One branch runs on the robot, full of small corrections nobody folded back; the other runs in simulation and holds the tests. Every fix is applied twice, and the second application is always wrong in some small way that costs an afternoon.

Half your team wants to abandon the simulator and work on the machine, because that is where the truth is. The other half points out that there is one robot, four engineers, and a knee actuator that costs real money whenever somebody is careless.

And underneath the argument is a question nobody has answered: when the simulated robot and the real robot disagree, which one is wrong, and how would you tell?

Should your team develop in simulation first?

Yes, almost every team should start in simulation, because early in a project the mistakes come faster than hardware can absorb them. In the first weeks your software is wrong in obvious ways — a sign flipped, a frame confused, a limit missing — and each of those on hardware means a collision, a damaged part, and a day of repair. In simulation each costs a restarted process, and that ratio is the whole argument. What makes the decision hard is that it does not hold. As the obvious mistakes get fixed, the remaining problems move into exactly the region the simulator models worst: friction, compliance, backlash, timing under load, sensors that degrade rather than fail. At that point the simulator stops catching your bugs and starts generating its own, and the team stays in it out of habit and because the tests live there. The real decision is not whether to simulate but when simulation stops paying, and that transition is quiet. You notice it in retrospect, usually while explaining why the demo went differently on the machine.

What does simulation-first development actually mean?

It means writing and testing the robot's software against a modelled world before running it on the machine, and it covers three practices teams routinely confuse. The first is simulation as a development environment: your code runs without hardware, so nobody waits for the robot. The second is simulation as a test harness: scenes and scenarios run automatically on every change, so you learn today that yesterday's fix still holds. The third is simulation as a data factory: you generate training episodes for a learned policy at volumes no physical robot could produce. These three have different lifespans. The development environment matters most early and fades as hardware arrives. The test harness gains value over time and should outlive the project. The data factory is either central to your approach or irrelevant, depending on whether you train policies. Teams get into trouble treating these as one activity: when the development environment stops paying they abandon the test harness with it, losing the only part still working.

What are your actual options for splitting simulation and hardware?

There are seven realistic arrangements, and what separates them is not which simulator you run but how much of your software changes when the robot arrives. You can work purely in simulation with no hardware plan. You can simulate first and add hardware in the loop later. You can run simulation and hardware bring-up as parallel tracks with different people. You can go hardware-first and keep simulation only as a regression suite. You can use simulation purely to generate training data. You can adopt the vendor's simulator for the robot you bought. And you can arrange the whole thing so the simulator and the robot feed the same message layer, which is where the middleware choice shows up: ROS 2 has the deepest simulator integrations in robotics and the largest set of existing bridges, while HORUS is an open-source real-time robotics middleware for Rust, Python and C++ where the three languages share the same shared-memory ring buffers, so messages between processes on one machine are not serialised and the program under test does not change shape when the data starts coming from a robot. Read the table as descriptions of arrangements, not a ranking.

What should a two- or three-person team do?

Simulate first, but keep one cheap physical machine on the desk from the beginning, however crude. A small team cannot afford parallel tracks, and its failure mode is not choosing wrong — it is choosing simulation, staying too long, and finding the gap at the worst moment. The corrective is earlier hardware in a smaller form. A hobby arm, a wheeled base, or a single actuator on a bench gives you a place to check the assumptions your simulated world is built on, and it costs a fraction of the real platform. Run the same software against both, keep them in one branch, and make it uncomfortable to let the physical path rot. You also cannot afford a dedicated simulation owner, which is preferable anyway: the engineers writing control and perception should maintain the scenes, so a lying model is corrected the week it lies. On timing the switch, see when to stop simulating and build the real thing.

What hardware do you need before simulation stops being enough?

You need whichever piece of the machine your uncertainty lives in, which is usually the actuator and its controller rather than the whole robot. Teams overestimate this and wait for a whole platform when one joint would have answered the question. If your doubt is how the arm behaves on contact, one motor with a force sensor on a bench settles more than a month of simulated touching. If your doubt is whether the onboard computer can carry perception and control together, you need that computer and nothing else. If your doubt is the sensor, buy it and point it at your actual environment, because the difference between a modelled camera and a real one in a room with windows is not subtle. Order early regardless of readiness, because parts take longer to arrive than anyone plans for. The cheapest mistake here is buying a partial rig too early; the most expensive is learning in month five that the model was wrong in month one.

What if you have a demo in six weeks?

Then decide immediately whether the demo happens on the machine or on a screen, and commit, because the hybrid is what fails. A simulated demo is legitimate for early-stage work when presented honestly. If that is the choice, spend the six weeks on behaviour and scene and do not touch hardware. If the demo must be physical, stop adding capability now, move everything to the machine, and use the remaining weeks on what simulation never covers — cabling that survives motion, an emergency stop that works, a startup sequence somebody else can run, recovery from the failure you know is coming. Reserve at least a third of the time for that, because it always takes it. Attempting both in six weeks produces two half-finished demonstrations and a team that has not slept. If you are behind, the physical demo with less capability beats the impressive simulated one, because the audience question is always whether the machine is real.

What if nobody on the team has shipped a robot before?

Then simulate first, and treat your first hardware contact as an experiment about your assumptions rather than a milestone. A team without shipping experience does not know which of its beliefs about the machine are wrong, so it cannot know where to look. Simulation is a good place to build competence in software structure, coordinate frames, and testing discipline, none of which need a robot. What simulation cannot teach is what goes wrong on real machines — connectors working loose, ground loops, a sensor reporting fine while quietly drifting, a motor that behaves differently warm than cold. So plan the encounter deliberately. Pick a week, put the software on the smallest real machine you have, and write down every difference you find. That list is your real education. Expect it to be longer than you thought, and expect one item on it to invalidate something you built for a month. A useful companion read is what is sim-to-real and why does it keep failing.

What does simulation-first look like when it goes wrong?

It looks like two robots: the one in the simulator, which works, and the one in the lab, which is behind, and a team that keeps choosing to improve the working one. The symptoms arrive in order. First a fix lands on the machine and is not folded back, because the simulated version needs it elsewhere. Then the branches diverge enough that running the simulation suite against the hardware code stops meaning anything. Then someone proposes improving the model to close the gap, and the model becomes a project — weeks of tuning friction and inertia to match a video. That work feels productive and produces nothing a customer sees. Finally the simulator becomes a second product with its own bugs, maintainer, and backlog, while the actual robot is developed by two people in the corner. The tell is the ratio of hours spent making the model truthful against making the machine capable. When the first exceeds the second for more than a couple of weeks, the practice has inverted.

What do teams try first, and why does it stop working?

Most teams try to build an accurate model of their robot first, and it stops working because accuracy in the model is not what determines whether behaviour transfers. The reasoning seems sound: the gap comes from the model being wrong, so make the model right. The team measures link masses, tunes friction, adds motor curves, matches a recording. Each step improves fidelity and none fixes the real problem: the software has learned to depend on values it will not encounter. A controller tuned against precise simulated friction is tuned against a number, and the real one differs, differs again when the joint is warm, and changes over the machine's life. Teams that transfer well do the opposite: they make the simulation less certain, not more accurate. Masses vary between runs, delays are inserted, sensors get noise. Software that survives a world that will not hold still tends to survive the real one, and the effort goes into the software, not the model. On how much fidelity is worth buying, see how realistic your simulation actually needs to be.

What do you give up by developing in simulation first?

You give up calibration on the things no model contains, and you give up it silently, which is what makes the cost hard to see. A simulator models the physics somebody thought to include. It does not model a connector that loses contact when the arm passes through one pose, a camera that changes exposure when the robot turns toward a window, a battery whose sag changes how motors respond late in a run, or heat. It also does not give your team the instincts that come from working near a machine that can hurt somebody: where to stand, what to do when it behaves oddly, how to bring it up safely twice a day. Teams that simulate for a long time build software that is correct against a specification and unprepared for a world. The second cost is schedule honesty. Progress in simulation is smooth, which implies a predictable finish, so plans get made on it. Hardware progress is lumpy, and the lumps arrive after the plan is committed.

When is ROS 2 the better choice?

ROS 2 is the better choice for teams whose workflow depends on existing simulator integration, and that is a large share of robotics teams. The connections between ROS 2 and the major simulators are mature and used by thousands of projects: description formats both the simulator and the control stack read, sensor plugins publishing in shapes the ecosystem expects, and a control interface that lets the same code drive simulated joints and real ones. Recording a run and replaying it into your software is solved there and unsolved almost everywhere else, and replay is how teams debug something that happened once. If your plan is to use an existing navigation stack, an existing manipulation planner, and an existing simulator bridge, ROS 2 is the answer and adopting HORUS instead would mean rebuilding integrations you could have had for free. The case for a shared-memory message layer arrives later and narrower: when several programs in different languages are competing for one onboard computer and the copying between them has become the thing limiting what the robot can run.

Will a more realistic simulator close the gap to the real robot?

No, and here is why: the gap is not mostly a rendering or physics-accuracy problem, it is a problem of unmodelled variation. Better graphics help perception models failing on appearance, and better contact solvers help manipulation research, so realism is not worthless. But teams that upgrade simulators expecting transfer usually find the same behaviours failing for the same reasons, because what breaks on the machine is rarely what the new simulator improved. The real robot has a loose belt, a warm motor, a delay that varies with load, and a floor that is not level. None appear in a fidelity comparison. The teams whose software transfers are not those with the most realistic simulator; they are the ones whose software never got to rely on any value being true. That is why deliberately randomising the simulated world outperforms perfecting it, and why a modest simulator used with discipline beats an expensive one used as a source of truth. Spend the budget on variation and on getting to hardware sooner.

Is simulation-first just a way to avoid buying hardware?

Partly, but not the way you think. Cost avoidance is a genuine and honest reason to simulate, especially for legged robots and manipulators where a fall breaks parts that take weeks to replace. But teams that simulate purely to defer hardware spending tend to spend the money anyway, in engineering hours, on a model standing in for a machine they will buy regardless. The stronger reason to simulate has nothing to do with cost: it is that simulation lets you run the same scenario many times, which hardware cannot do. You can put the robot in the exact failing state again, change one thing, and watch — a debugging capability, not a savings measure. Framed that way, the decision changes: keep the scenarios you need to repeat and the ones that would damage something, and stop maintaining everything else. That is a much smaller simulation than most teams build, and it is still there two years later when a regression appears and somebody needs to reproduce it.

How do you decide how much of your work to simulate?

Decide by what you need to repeat, not by what you can model. Sort your open questions by where each can be answered: some need a physical joint, some need a repeatable scenario, some need only a unit test with no world at all. The middle group is your simulation scope, and for most teams it is smaller than what they built. Then set a review point — a date, not a milestone — and ask one question: in the last few weeks, did the simulator find a bug in our software, or only create work maintaining itself? A simulator that has stopped finding bugs has finished its first job and should shrink into a regression suite. Keep the scenarios that already broke the robot once, delete the scenes nobody runs, and move the hours to the machine. Repeat the review, because the answer changes as the project matures.

Decide by situation rather than by preference:

• If you have no hardware yet -> simulate everything, because software wrong in obvious ways is cheaper to fix without a robot in the room.
• If you are two or three people -> simulate first with one cheap machine on the desk, because you cannot afford two tracks or a late surprise.
• If your product depends on contact -> get to hardware early, because that is where models are least honest.
• If you train learned policies -> simulate heavily and randomise hard, because variation is what transfers rather than fidelity.
• If the robot already exists -> keep simulation only as the suite that catches regressions, because the machine is the better source of truth.
• If a physical demo is weeks away -> stop adding capability and move to the machine, because bring-up always takes the time you did not budget.

When two arrangements stay close, weigh the tools underneath them against the five axes of the HORUS Fit Framework: ecosystem size, setup effort, team size fit, deployment target, and licence. Take the one that loses on the fewest — five honest questions about your team rather than scores about the software. If your simulator and your robot need to feed the same programs on one onboard computer, star HORUS on GitHub so it is in your list when you start building.