Do You Need Continuous Integration for Robot Software?

Yes: build and check every merge automatically, because manual bench testing only catches what somebody remembered to try. The rig is the expensive half and can wait; the checks that pay first are the dull ones that run without being asked, on ROS 2 or on a smaller stack such as HORUS. That flips for a solo builder still deciding what the robot is. The rest of this post is for a team that shares a repository, argues about this in planning, and has to justify the setup time to somebody who counts hours.

The argument starts in a retrospective and never finishes. Somebody says the robot broke on Tuesday because of a change nobody tested. Somebody else says of course it did, we test on the robot, and on Tuesday the robot was in pieces on a workbench with its arm off. Both statements are true. Neither of them suggests what to do on Wednesday.

Meanwhile the symptoms stack up. A branch that worked on one laptop does not compile on the computer that ships inside the machine, and nobody finds out until the morning of a demo. Two people fix the same startup bug in different ways in the same week. The Python side and the C++ side drift apart, because only one of them gets exercised before a run. There is a test folder in the repository, last touched in March, that nobody can run any more because it depends on a sensor now bolted to a different robot.

And then the sentence that ends every one of these conversations, said with a shrug because it sounds unanswerable: we would automate this, but our tests need the robot, and we only have one robot.

Do you need continuous integration for robot software?

Yes, from the moment a second person commits to the repository, and the version you need first is far smaller than the one the phrase suggests. It is not a wall of tests. It is one machine that rebuilds the code for the computer that goes inside the robot every time somebody pushes, and tells the person who pushed when the rebuild fails.

That single check deletes an entire category of bad day. Nobody discovers on demo morning that a dependency existed only on one laptop. Nobody spends an afternoon learning that the branch merged last week never compiled for the target board at all, only for a developer's desktop. Nobody argues about whose change broke the build, because the answer arrives addressed to the person who made it.

The question teams usually mean is a different one: do we need the elaborate version, with a hardware rig, recorded runs and a dashboard nobody reads. That answer is no, not yet, and quite possibly not this year. The two halves of automation have very different prices, and teams that argue about the expensive half end up with neither half. Start with the part that needs no robot.

What does continuous integration actually mean for a robot?

It means a computer somewhere rebuilds your software and runs a fixed list of checks every time somebody pushes a change, with nobody remembering to start it. That is the same idea used in every other kind of software. What differs for a robot is the list.

For a web service the list is unit tests, finished in the time it takes to fetch a coffee. For a robot the list splits into three layers with very different prices. The first layer needs nothing physical: does the code build for the board that ships inside the machine, does the configuration still load, do the message shapes still agree between the Python process and the C++ one. The second layer needs a faked or simulated world: started headless, does the software reach a running state, does the arm stop before it hits the table, does the safety boundary hold when the input is nonsense.

The third layer needs an actual robot, or the electrically interesting parts of one, wired to a machine that runs overnight. Teams picture the third layer, price it, wince, and do nothing. Almost all of the value sits in the first two.

What are the actual options for testing robot code automatically?

There are about six, and they stack rather than compete. Bench testing only, where a human drives the machine and watches it. Build checks, which compile every language in the repository for the target computer. Behaviour checks in a headless world, where the software runs against faked or simulated inputs and something asserts what the robot did. Replay checks, where a recording of a real run is pushed back through the software and compared with what happened that day. A hardware-in-the-loop bench, permanently wired and always available. And a canary unit in the field that takes a release before the rest of the fleet.

What you build on changes the price of each rung. A ROS 2 project inherits an ecosystem where recording, replaying and launching a set of processes are solved problems that other people maintain and document. A project built on a middleware such as HORUS, where Rust, Python and C++ processes pass messages through shared memory on one machine, has a different advantage: every process is an ordinary program in its own language, so the normal test runner is the entire apparatus and the automation server never needs a robot attached to it.

How do the testing options compare for a robot team?

The comparison is mostly about what each option needs before it can run at all, and how loudly it fails when it breaks. Read the last two columns first, because that is where teams spend money they did not plan to spend.

No row wins outright. The rows are a ladder, and most teams should stand one rung higher than they currently do.

Does a two-person team need continuous integration?

Yes, and a pair gets more out of the first rung than a large group does, because there is nobody else around to notice breakage on your behalf. In a group of twenty, a broken build is found within the hour by whoever pulls next. In a pair, a branch stays broken for a week, because each of you only ever builds the part you are working on that week.

The second reason is memory. A pair carries an enormous amount of unwritten knowledge about which order things must start in, which environment variable matters, and which board needs the old driver. Automated checks are how a small team writes that down without ever sitting down to write documentation. The check either passes or it does not, and the file that defines it is the honest version of the setup instructions.

The third reason is the holiday problem, covered from another angle in what suits solo builders and pairs. What a pair should not do is build the expensive rungs. A bench rig maintained by two people becomes a third project that neither of them has time for.

Does your hardware make automated testing impractical?

Sometimes, and the fix is to stop trying to test the whole robot at once. Hardware gets in the way in three specific ways, and each has a different escape. The computer inside the machine is not the computer you develop on, so a build that succeeds on a laptop proves less than it appears to. That one is solved by building for the target architecture on the automation server, which costs an afternoon of setup and then never bothers you again.

The sensors are not present, which blocks behaviour checks rather than builds. That is solved by deciding, deliberately, what stands in for each sensor, which is a decision with real consequences and its own trade-offs in simulating sensors versus faking them.

The third obstacle is that the machine can damage itself when a test does something stupid at three in the morning. That is why the hardware rung usually starts as a robot with its wheels off the ground, or a single joint on a bench, rather than the whole machine free to move. A partial rig that runs unattended beats a complete one that needs a supervisor.

Should you skip continuous integration to hit a demo date?

No, and the reason is that the cheap rung is what protects the demo. A deadline is precisely the situation where people merge quickly, test the part they changed, and assume the rest still works. That assumption is what fails on stage. The build check costs an afternoon once and then keeps you from the specific failure that ruins demo mornings, which is software that never ran on the target computer at all.

Skip the expensive rungs without guilt. Under deadline pressure, a hardware bench, a replay corpus and a canary unit are all projects competing with the robot for the same hands. Postpone them by name and write down when you will revisit the decision, so it is a choice rather than a drift.

There is a version of this that goes wrong. Teams under pressure sometimes add checks that are slow, flaky, or that fail for reasons nobody can explain, and then start clicking merge anyway. That is worse than having nothing, because it teaches everyone to ignore a red light. If a check cannot be trusted before the deadline, delete it before the deadline and add it back afterwards.

What if nobody on the team has set up continuous integration before?

Then you are about half a day from the first useful version, because that version is one file describing the commands you already type by hand. Install the dependencies. Run the build. Run whatever tests exist. That is a complete and genuinely valuable setup, and every hosted service will run it on a fresh machine without a robot in sight.

The part that surprises newcomers is not the syntax. It is that a fresh machine exposes every undocumented step your laptops have accumulated: the package installed manually last year, the environment variable in a shell profile, the directory that has to exist before anything starts. The first attempt fails several times for boring reasons, and each failure is a piece of tribal knowledge becoming written knowledge. That process is unpleasant and worth it.

The trap is scope. A first-time team reads about test matrices, containers, artefacts and staged deployments, decides the whole thing is too big, and ships nothing. Build one check. Watch it turn red once for a real reason. Add the second check only after that.

What does it look like when a team has no automated checks?

It looks like the same three bugs returning every few months, each time discovered by a person rather than a machine. Somebody fixes the startup ordering problem. Two months later a refactor reintroduces it, nobody notices, and it surfaces during a customer visit as a robot that sits still and blinks.

The second signature is the manual checklist. Teams without automation always invent one: a wiki page listing the steps to verify before a release, twenty items long, followed carefully the first three times and skimmed forever after. The checklist is not wrong. It is simply a program written in the wrong language, executed by tired humans, at the exact moment when everyone is in a hurry.

The third signature is the phrase works on my machine, said without irony, because it is true and also useless. A robot team is unusually exposed to that sentence, since the machine that matters is not anybody's laptop. When the definition of working lives in a person's habits instead of in a command anyone can run, onboarding a new engineer takes weeks and every release is a small act of faith.

What do you give up by putting robot code behind automated checks?

You give up speed on the individual merge, and you take on a system that itself needs maintenance. Both costs are real, and pretending otherwise is how teams end up resenting the setup they asked for. A check that takes twenty minutes changes how it feels to fix a typo, and people will route around it if the wait feels arbitrary.

You also give up a certain freedom to change your mind. Tests encode current behaviour, and a large suite written against a design you are about to replace becomes an argument against replacing it. That pressure is subtle and worth naming early, because it shapes architecture decisions long before anybody notices.

The third cost is upkeep. Runners expire, dependencies shift, a hardware bench needs somebody to unplug it and plug it back in. Automation is a small internal product with users, and internal products without an owner rot. Budget a named owner from the start rather than discovering the need after everyone has learned to ignore a permanently failing check that nobody can remember the purpose of.

When is ROS 2 the better choice?

ROS 2 is the better choice whenever the thing you want to test automatically is the whole robot rather than your own handful of processes. If a meaningful check means bringing up drivers, a simulated world, a navigation stack and a recorded run, ROS 2 already has all of those pieces, along with the launch descriptions that start them together and the file formats that let a failure from the field be replayed at a desk.

ROS 2 also wins on the boring axes that decide projects. Contractors arrive knowing the tooling. Vendors ship drivers for it. Public examples of testing setups exist to copy, and copying is the fastest form of learning an unfamiliar practice.

HORUS is not the answer in those cases. A shared-memory middleware carries messages between processes on one machine and does not bring you the ecosystem's drivers, its simulator bridges or the years of shared tooling built around recorded runs. If your automated checks depend on those, choosing something smaller means writing your own test scaffolding, which is a second project nobody funded.

Is continuous integration overkill for a robot only one person touches?

No, and here is why. The colleague you are protecting is yourself in four months, after a break from this code, holding a machine that will not start and no memory of which step is missing. Automated checks are the only form of documentation that cannot quietly go out of date, because the moment it stops being true it turns red.

There is a second reason that matters more for solo work than for teams. Working alone, there is no second opinion, so a bad assumption can live for months without ever meeting resistance. A build that runs on a clean machine is a cheap second opinion. It has no politeness and no context, and it will tell you that your project depends on something you installed once and forgot.

The third reason is what happens when the project stops being solo. A single robot with one owner becomes a demo, then a pilot, then a thing somebody else must run. The checks you wrote while alone are what make that transfer possible without a week of shoulder-to-shoulder explanation.

Does automated testing mean you can stop testing on the real robot?

Partly, but not the way you think. Automation does not reduce how often you touch the machine. It changes what those sessions are for. Without checks, bench time is spent rediscovering whether the basics still work: does it start, does it connect, does the arm move at all. With checks, the basics are already known, and bench time goes to the things only a real machine reveals.

Those things are stubbornly physical. A connector loosening under vibration. A camera exposure that behaves differently in afternoon sun. A motor drawing more current when warm. Timing that holds on a desktop and slips on the board in the robot. None of that is visible from a test runner, and no simulated world contains it, which is the whole subject of what to do when simulation and hardware disagree.

The correct expectation is a division of labour. Automation guards against regressions, meaning things that used to work. Bench sessions explore, meaning things that never worked because nobody knew they were a question. Automating away the first kind is what buys the time to do the second properly.

How should a team decide what to automate first?

Write down the last five things that broke, then automate the cheapest check that would have caught the most of them. This takes one meeting and beats any general advice, including this article, because your five are not somebody else's five.

The list usually sorts itself quickly. If most entries are build and environment failures, you need one build check and nothing else this quarter. If most are behaviour failures, meaning the robot did the wrong thing rather than failing to start, you need a headless check with a faked world, and the work is in deciding what stands in for each input. If most were only ever seen once, on the real machine, in a place you cannot revisit, your first investment is recording, so that next time the failure comes home with you.

Then set a rule about red lights and keep it. A failing check either gets fixed or gets deleted the same week. Anything else and the whole setup quietly becomes decoration, which is the state most abandoned automation reaches within a year.

• If you are two people with a shared repository -> add build checks this week, because the first rung costs an afternoon and pays back on the first broken merge.
• If your robot decides things rather than only moving -> add headless behaviour checks next, because logic errors are cheap to catch and expensive to explain in front of a customer.
• If the same bug keeps returning after being fixed -> start recording runs, because a recording turns an unrepeatable failure into a check that runs forever.
• If units are going to buildings you cannot visit -> build a hardware bench, because the alternative is discovering the problem through somebody else's complaint, as covered in managing many deployed units.
• If you are one person and the robot is still an idea -> skip all of it for now, because the design changes faster than any check can keep up with.

The HORUS Fit Framework reduces the surrounding stack choice to five axes: ecosystem size, setup effort, team size fit, deployment target, and licence. Score each option on all five, because the automation you can sustain depends heavily on how much of the robot you did not write.

If the axis that keeps deciding things for you is timing between your own processes, put HORUS on your reading list rather than this quarter's roadmap: star it so it is in your list when you start building.