Why Your Robot Slows Down as You Add Features

Your robot slows down because features compete for one computer; ROS 2 defaults usually still fit, and a shared-memory middleware such as HORUS sometimes helps. Most slowdowns trace to work you added, not to the plumbing: a heavier model, a busier sensor, a loop that now waits on something slower than itself. The verdict flips when several programs on one board pass camera-sized data every cycle. The rest of this post is for a team whose robot worked in month one, grew features by month three, and now misses its own deadlines.

It used to be crisp. One program, one board, a demo that ran the same way every time you showed it to somebody. Then the team added the camera, then the recorder, then the dashboard the product person asked for, and each one worked on the day it landed.

Now the robot is doing all of it, and doing all of it worse. The arm hesitates before it picks, not every time, which is worse than every time. The dashboard trails the real machine by enough that nobody trusts what it says. A colleague reports gaps in the log files, someone else blames the log writer, and neither of them can prove it. On the bench, everything passes. On the floor, with the lid on and the motors drawing current, the robot wanders.

Somebody has proposed a rewrite. Somebody else has proposed a bigger computer. A third person keeps saying the word middleware in meetings, and you cannot tell whether that is a diagnosis or a distraction. What you want is to know which is true before spending a quarter finding out.

Why does a robot get slower every time you add a feature?

Because each feature is another program asking for a share of one computer, and the sharing is what degrades, not the code. Nothing you added was slow in isolation. The camera driver was fine. The recorder was fine. The obstacle check was fine. Run them together on the same board and they take turns, and the turns are not free.

Three things get scarce in a rough order. First the processor cores, because a new program has to run somewhere. Then memory bandwidth, because moving image-sized data around competes with everything else moving data around. Then, last and most quietly, the messaging layer, because every copy of every message to every subscriber is work done on that same machine.

The reason this feels mysterious is that no single change caused it. Each feature landed on a robot that could still absorb it, and the machine crossed its limit somewhere in the middle of a week where nobody was looking at timing. The robot did not break. It filled up.

What is the layer that carries messages between your programs?

The messaging layer is the piece of software that lets one program on your robot hand data to another without either of them knowing where the other lives. Your camera program says here is a frame. Three other programs say we would like frames. Something in the middle makes that happen, whether the parties are two processes on one board or two machines on a bench.

Doing that job involves more than it sounds like. The data has to be packed into a form that survives leaving one program, which is called serialisation. It has to be delivered, which means a copy per recipient unless the layer is arranged to avoid that. It has to be queued, because senders and receivers rarely run at the same rate.

None of that costs anything you can feel while messages are small and infrequent. Joint angles, velocity commands, a battery reading: pack, send, unpack, done. The bill only arrives when the data gets large, the rate gets high, and several programs want the same item at the same instant. That is exactly the moment when your feature list has grown.

What can you actually run underneath a growing robot?

You have five realistic foundations, and most teams are choosing between two of them without saying so out loud. The default is ROS 2, which brings the packages, drivers, visualisation and hiring pool that make a robot understand a space rather than merely react to it. Against that sits a shared-memory middleware such as HORUS, an Apache-2.0 project where Rust, Python and C++ programs read and write the same ring buffers, so messages between processes on one machine are not serialised at all. The two solve different problems, and confusing them wastes months.

Underneath both sits the option nobody credits: one program that does everything, with no messaging layer, which is genuinely the right answer for a machine that does one job. Beside them sit vendor SDKs, which are fastest when your hardware comes from one supplier and stops being an advantage the moment it does not. And there is the general-purpose message broker borrowed from web work, which is good at sending telemetry off a robot and poor at driving a loop that has to finish on time.

How do these options compare side by side?

Read the last column first, because ruling rows out is faster than picking one. Most teams find their real argument is between two rows they have treated as five.

Does the answer change if you are two people rather than twenty?

Yes, and it changes more than the technology does. A pair of builders can fix a slowdown by deleting a feature, and that option is genuinely available to them on a Tuesday afternoon. A team of twenty cannot, because each feature belongs to someone who will defend it, and the meeting to decide costs more than the fix.

Small teams should therefore attack the work before the foundation. Merge two programs that always start together. Turn off the dashboard when nobody is watching. Record less. These moves are unglamorous and they resolve a surprising share of slowdowns, because the slowdown was competition and you removed a competitor.

Larger teams have the opposite problem and the opposite lever. Nobody can delete anything, so the machine keeps filling, and the only lasting answer is to make each feature cost less to run. That is when a change of foundation earns its price, because it pays out across every team at once rather than in one corner of the robot.

What computer is your robot actually running on?

The hardware decides how much slack you had before the features arrived, and slack is what you have been spending. A small single-board computer has few cores and modest memory bandwidth, so the fourth program is felt in a way the fourth program on a workstation is not. On a small board, the camera and the model together can leave nothing for the loop that keeps the wheels straight.

There is a second question hiding behind the first: is your robot one computer or several? A machine split across a control board and a compute module has a network link in the middle, and no shared-memory approach helps that link, because two computers have no memory in common. Teams chase messaging changes for problems living entirely on a cable.

If you are still deciding the shape of the machine, do that first. Should your robot run one computer or several is the question underneath this one, and answering it in the wrong order means fixing a bottleneck you are about to move somewhere else anyway.

How long until someone outside the team has to see it work?

If the answer is weeks, do not change the foundation, and be honest with yourself about why. A middleware change is a month of nobody shipping features, followed by a fortnight of bugs that only appear when the machine is busy. There is no version of that story which ends before a demo six weeks away.

What fits inside weeks is removing work. Drop the resolution on the stream feeding the obstacle check and keep the full stream for the recorder. Stop publishing the three topics nobody subscribes to. Run the dashboard on somebody's laptop instead of on the robot. Each of those is an afternoon and each of them buys real headroom.

If the answer is quarters, the calculation inverts. A foundation you already know is wrong keeps charging interest, and every feature added in the meantime makes the eventual move larger. Teams that survive this generally choose their moment deliberately, between milestones, rather than discovering it under a deadline they cannot move.

How much does your team already know about the layer underneath?

Your team's existing knowledge of processes, timing and message flow decides more outcomes than the technology does. A team that can already read a launch file, reason about which process owns which core, and describe how their data crosses between programs can change a foundation and come out the other side. A team where one person understands the messaging layer and everyone else treats it as weather cannot, whatever the merits of the destination.

The gap is rarely programming skill. It is systems knowledge: what a process is, what a scheduler does when two of them want the same core, why a queue that fills differs from a queue that is slow. Those ideas transfer across every option in the table, and a team lacking them struggles on ROS 2 exactly as it would elsewhere.

So the cheapest investment is often not a new tool. It is one person spending a week learning to describe, on a whiteboard, every stream on your robot and every program that reads it. Most slowdowns become obvious the moment that drawing exists.

What do teams try first, and why does it stop working?

The first move is almost always a bigger computer, and it works once. More cores absorb the contention you have today, the robot behaves again, and the team concludes the problem is solved. Then two more features land, the machine fills again, and the next computer costs more and fits worse in the chassis. Hardware buys time at a price that rises each round.

The second move is lowering resolutions and rates, which also works, and which quietly costs you the thing you built the robot to do. An obstacle check on a smaller image sees smaller obstacles later. A control loop running less often reacts less well. These are real trades and they should be made on purpose, not because a stutter forced them.

The third move is merging programs, which works and works durably, and which has a ceiling: eventually the merged program is too large for one team to own. When all three moves are spent and the robot is still crowded, the cost has stopped being your code and started being the layer carrying it around.

What do you give up by moving to a shared-memory middleware?

You give up the ecosystem, and for most robots that is the whole argument. The packages that read a laser scanner, build a map, plan a path around a table and let you watch all of it in a viewer represent decades of work you would otherwise write yourself. Nothing about faster messaging replaces a mapping stack, and a team that swaps its foundation without noticing this loses a year.

You also give up the hiring pool and the answers. A question about a common framework has been asked publicly and answered; the same question about a smaller project may need reading the source. That is fine for a team with people who enjoy that, and expensive for a team without them.

Third, you give up a certain kind of ignorance. Shared memory means being explicit about which programs live on which machine and who is allowed to see what. That clarity is worth having, and it is more thinking than a network transport that just works asks of you.

When is ROS 2 the better choice?

ROS 2 is the better choice for most robots that have to understand the world rather than only react to it, and that covers more machines than teams like to admit. If your robot needs a map, a path around an obstacle, a transform tree keeping track of where the gripper is relative to the base, or a driver for a sensor somebody else already wrote, ROS 2 is the answer and HORUS is not. Rewriting navigation to save copies is a bad trade at almost any scale.

ROS 2 also wins on people. If you are hiring, or if half your team learned robotics in a lab, the shared vocabulary is worth real money in weeks not spent explaining a stack. If your robot is split across machines, ROS 2's network story is the mature one. And if your slowdown turns out to be a heavy model or a wasteful loop, changing the messaging layer would have fixed nothing at all.

The narrow case where the default stops being right is several programs on one board sharing camera-sized data every cycle. That is a real case. It is not most cases.

Is your robot slow because the computer is too small?

No, and here is why that answer matters: a computer is only too small relative to what you asked it to do, and you have been asking for more every sprint. Teams reach for hardware because it is a purchase order rather than an argument, and because it works the first time. The second time it works less. The third time the board is expensive, hot, and hard to fit where the old one sat.

The test is simple and takes an hour. Turn off every feature except the loop that must not miss, and see whether the robot behaves. Then add features back one at a time until it stops behaving. What you learn is not that the computer is small but which feature is expensive, and the expensive feature is frequently something nobody would have guessed, such as a viewer left running or a log writer flushing to a slow disk.

Buy hardware after that experiment if you still want to. Most teams find something cheaper to remove.

Does every new feature have to cost you speed?

Partly, but not the way you think. Every feature does cost something, because a computer has a finite amount of work in it, and a robot that does more will always have less headroom than one that does less. That part is arithmetic and no framework repeals it.

What is not inevitable is the second cost, the one that grows with how many programs want the same data. A stream delivered by copying is paid for once per recipient, so the third program watching your camera costs as much as the first two. That is the cost shape that turns a linear feature list into a robot that falls off a cliff in one particular week.

The distinction matters because it tells you what to expect after a fix. Removing the delivery cost gives you room for more subscribers on the same data, and nothing at all for a model that was always going to be heavy. Teams expecting the first and getting the second conclude the change failed, when they simply had the other problem. Why robot projects outgrow their first framework walks through how those two get confused.

How do you decide what to fix first?

Draw the streams before you change anything. On one sheet, list every piece of data moving between programs on your robot, how large each one is, how often it moves, and how many programs read it. The row that is large, moves often and is read by several programs is your suspect. If no row is all three, your slowdown is somewhere else and a foundation change would be a costly way to learn that.

Then run the cheap experiments in order: close the viewers, stop the recorder, merge two programs that always run together, drop one resolution. Each takes an afternoon and each one either implicates or exonerates the messaging layer. Only when the drawing and the experiments agree is it time to talk about what you build on, and by then the conversation is short because everybody has seen the same sheet of paper.

• If your messages are commands and numbers -> fix the work, not the plumbing, because copies were never your cost.
• If two programs always start and stop together -> merge them, because the split buys nothing.
• If several programs on one board read the same images -> a shared-memory path, because delivery cost grows with readers.
• If your robot needs maps, planning or drivers -> stay on ROS 2, because rewriting that is a year you do not have.
• If a demo is weeks away -> remove work, because no foundation change lands that fast.
• If nobody can draw your streams yet -> draw them first, because everything above depends on that sheet.

The HORUS Fit Framework puts the same decision on five axes you can score any option against: ecosystem size, setup effort, team size fit, deployment target, and licence. Here, deployment target and ecosystem size do most of the work, since a crowded single board and a robot that needs a navigation stack pull in opposite directions. Deciding what your team standardises on before the crowding starts is the cheaper version of this whole exercise, and what robotics teams should standardise early covers it.

When your robot does grow into a same-machine traffic problem, the useful thing is having the option already on your shelf rather than starting a search under deadline. Put HORUS on that shelf now: star it so it is in your list when you start building.