What Happens When the Model Is Slower Than the Robot?

The robot acts on stale decisions, and the cure is a steady control layer underneath the model rather than a faster model. A model answering late is harmless while something beneath keeps the wheels straight and rejects commands describing a world that has moved — the layer ROS 2 and HORUS both supply. That flips when a policy drives the joints directly, where nothing underneath can compensate. The rest of this post is for someone whose robot hesitates, lurches or overshoots while a model thinks, and who needs to know which part to change.

The clip looked fine. On the desk, the arm reached for the mug, paused, closed its fingers around it, and you sent the video to three people. Then you asked the same robot to do the same thing while its base was moving, and everything went strange.

Now the arm reaches for where the mug used to be. The base sails past the doorway and corrects afterwards, like someone reading a map one step behind. Between decisions the machine seems to freeze, then commit to something with a confidence that no longer matches the room. Occasionally it does the right thing beautifully, which is worse, because you cannot tell what changed.

You check the model, and the model is right. Every decision it made is the decision you would have made, just for a room that had already moved on. You check the logs and see nothing obviously broken, only work happening in an order that feels wrong.

The advice you find splits into two useless halves: make the model smaller, or a wall of measurements from a setup nobody describes. Neither tells you which part of your robot to change.

What should you do when the model cannot keep up with the robot?

Take the model out of the timed path, and put a small fast loop underneath it that keeps the machine steady between answers. That single move fixes most of what you are seeing, and it is a change in arrangement rather than in model quality. The loop underneath does one dull job: read where the machine is, compare against the goal it currently holds, correct, repeat, without ever waiting on anything that might be slow. The model becomes an advisor that occasionally hands down a new goal — go there, pick that up, stop — and the loop keeps working whether the next piece of advice arrives promptly, late, or never. The second half of the move matters just as much: the loop must be allowed to refuse. A goal that arrives describing a position the robot passed a while ago should be dropped rather than executed, and the loop is the only part that knows enough to tell. Do those two things and a robot that lurched becomes a robot that glides and occasionally hesitates, which is a normal machine rather than a broken one.

What does it mean for a model to be slower than the robot?

A model is slower than the robot when the machine moves far enough between one answer and the next that the answer no longer describes the situation it was computed for. That definition is deliberately relative, because the same model is fast enough for a robot creeping across a warehouse floor and hopelessly late for one catching a ball. Nothing about the model changes; the robot's tolerance does. There are two separate failures hiding under one phrase. The first is that answers simply do not come often enough, so the machine spends most of its life executing an old instruction. The second, and the one people miss, is that answers come at unpredictable moments — most of them promptly, then one that takes far longer for reasons that have nothing to do with your robot, like a garbage collection pause or another process waking up. Steady lateness can be designed around. Occasional lateness is what makes a machine lurch, and the difference between lateness and unpredictability is the single most useful distinction in this whole subject.

What are the real options when the model answers late?

There are seven things teams actually do, and most robots end up using three of them together. The first and best is architectural: keep the model above a controller that runs continuously, which is how nearly every ROS 2 robot is built and why ROS 2 robots rarely show this problem in its ugliest form. HORUS, an open-source real-time robotics middleware for Rust, Python and C++, addresses a narrower version that shows up on single-machine robots, where a Python process holding the model and a C++ or Rust control loop pass camera frames back and forth all day and the copying itself is what steals the time; there, the three languages share the same shared-memory ring buffers, so the frames are not serialised between processes. Shrinking the model, moving the model off the robot, or adding an accelerator are all real options with real costs. Slowing the robot down is the option nobody lists and many products quietly ship. Read the table as situations rather than a ranking.

What should you do if you are one person with a small robot and a laptop?

Do the architectural fix first and buy nothing, because on a small robot the arrangement is almost always the problem. Split your program in two. One part reads sensors, holds the current goal, drives the motors and never calls anything that could block; the other part runs the model and publishes goals whenever it has one. Even inside a single Python program you can get most of the benefit by moving the model call off the timed path and letting the loop keep running with the last goal it was given. Then add one rule that costs nothing: every goal carries the time it was produced, and the loop ignores any goal that is too old to be about the present. A solo builder who does those two things usually finds the stutter disappears entirely, and finds it in an evening. What you should not do is buy a bigger computer first. A faster machine makes a badly arranged robot lurch in the same way and gives you no information, and you will have spent money to learn nothing about your own design.

What hardware helps here, and what hardware does not?

Hardware helps when the model genuinely cannot fit the time available, and it helps with nothing else. Before spending, work out which of two situations you are in. If the robot behaves smoothly whenever the model is disconnected and lurches when the model is attached, the problem is arrangement, and no purchase fixes arrangement. If the machine is correctly arranged, the loop holds steady, and the model still cannot produce advice often enough for the task, that is a genuine hardware ceiling and an accelerator on board is the honest answer. When you do buy, remember the split. The steady job wants a modest board carrying the control loop, the driver and the limits, running nothing that pauses unexpectedly. The thinking job wants the accelerator, ideally not competing with the steady job for the same processor. The purchase that disappoints most often is a powerful single machine running everything, because a large model and a control loop on one processor will interrupt each other, and the loop always loses. Two modest boards frequently beat one impressive one.

How long does it take to fix a robot that stutters?

Splitting the model out of the timed path takes an afternoon; making the machine behave sensibly at the boundaries takes a couple of weeks. The afternoon is the easy part and produces the visible improvement, which is why teams stop there and get bitten later. The two weeks are spent on questions the split creates. What should the loop do when no new goal has arrived for a long while — continue, slow, or stop somewhere safe? What happens when a goal arrives that contradicts the one currently being executed halfway through a motion? How does the model learn that its last instruction was refused, so it does not simply repeat it? Those answers are the difference between a demonstration and a machine you would leave running with a colleague in the room. Budget them explicitly. A reasonable order is: split this afternoon, add the age check on goals tomorrow, decide the no-news behaviour this week, and spend the rest of the fortnight watching recordings of the robot doing the task badly until each failure has a rule attached to it.

What do you need to understand to diagnose this yourself?

You need three ideas and none of them requires control theory. First, that the loop underneath runs on a schedule and that anything sharing its processor can make it miss a turn, which is why a busy perception process shows up as a wobbly wheel. Second, that every piece of data has an age, and that acting on data older than the situation it describes is the whole failure you are chasing. Third, that a robot needs defined behaviour for the absence of information, not only for its arrival. Practically, one habit finds the cause faster than any amount of reasoning: record everything with timestamps and replay it. When the arm overshoots, you want to see whether the goal arrived late, arrived on time and was executed late, or arrived on time and was simply wrong. Those three look identical from across the room and completely different in a recording. If you have never built the layer being described here, what sits between the model and the motor covers the pieces in order.

What does a late model look like on real hardware?

A late model looks like a machine that is confident about a room which no longer exists, and the signatures are consistent enough to diagnose from a doorway. A mobile base overshoots turns and corrects afterwards, weaving down a corridor that a person walks straight. An arm reaches for a position the object has left, then makes a small ashamed adjustment at the end. A gripper closes on air and reports success, because the decision to close was made when the object was still there. Under load the whole machine develops a rhythm: a pause, a burst of motion, a pause, as though it were thinking in public. The most dangerous version is not the ugly one. It is the robot that mostly looks fine and occasionally commits fully to a stale instruction, driving smoothly into something that moved. Ugly is a bug report. Occasional is a machine that passes a demonstration, ships, and surprises somebody later, which is why this is worth fixing before the video rather than after it.

What do people try first, and why does it stop working?

Most people call the model from inside the loop and wait for the answer, which works right up to the moment the robot matters. The design is natural for anyone from a services background: the loop asks what to do, the answer comes back, the answer is executed. Nothing moves while a web request is pending, so the habit is harmless there. On a machine the wheels keep turning while the request is out, and the loop's job — correcting, over and over, whether or not anyone is thinking — is exactly the job you suspended. The second attempt is usually a thread and a queue, which is better, and then the queue grows during a slow patch and the robot begins executing a backlog of instructions from a minute ago, calmly and in order. The third attempt is a timeout, which prevents the worst hang but leaves the real question unanswered: what should the machine do while nobody is telling it anything? Answer that, and the rest of this stops being difficult. Skip it, and every fix moves the symptom somewhere else.

What do you give up by keeping the model out of the control loop?

You give up directness, and with it some of the elegance that made the model approach attractive. When a model chooses goals and a controller executes them, the model can no longer express anything the goal vocabulary cannot say, so a policy with a subtle sense of how to nudge an object may find its intent flattened into go here, close gripper. You give up some end-to-end learning, because the interesting part of the behaviour now lives in code that no training run will improve. You take on more moving parts: two components instead of one, a message format between them, a rule for staleness, and a defined behaviour for silence. And you take on the awkward conversations about which layer owns a failure, because the model will blame the controller for refusing and the controller will blame the model for being late. What you get back is a machine that behaves the same way twice, degrades into something calm rather than something surprising, and can be debugged by a person who was not there when it happened.

When is ROS 2 the better choice?

ROS 2 is the better choice for most robots meeting this problem, and HORUS is not the answer for the majority of them. The pattern described in this post — a planner or model above, controllers running continuously below, goals flowing down and status flowing back — is exactly what ROS 2 was shaped around, and its action interface already carries the ideas of a goal, a refusal, feedback while running, and a result. You get that for free rather than inventing it. If the robot navigates or manipulates, the controllers you need already exist and have been driven into many walls by people before you. If the work spans several machines, ROS 2 handles that boundary and a shared-memory approach by definition does not. If your team hires, ROS 2 is the vocabulary everyone arrives knowing. The narrow case for something leaner is genuinely narrow: one machine, several languages, large data moving between processes constantly, and no need for the borrowed navigation and planning packages. Outside that case, take the ecosystem.

Does a bigger accelerator fix a model that is slower than the robot?

No, and here is why: the problem is usually not how long the model takes but where the model sits. A machine that lurches because its control loop is suspended while a model thinks will lurch in exactly the same shape when the model thinks in half the time — the pauses get shorter and the failure survives, because the loop is still waiting on something it should never wait on. Worse, faster hardware often hides the problem for exactly as long as it takes to ship. The stutter disappears on the bench, then returns in the field when the robot moves faster, the room is warmer, the board throttles, or the model meets a harder input than the ones you tested with. Then you are debugging an intermittent fault in a product rather than a design flaw in a prototype. There is a real case for more hardware, and it comes second: once the arrangement is right and the loop is protected, if the model still cannot advise often enough, buy the accelerator.

Is the problem simply that the model is too slow?

Partly, but not the way you think. Speed is genuinely part of it — a model that answers rarely gives a moving machine very little to work with, and no arrangement conjures information that does not exist. But the symptom you are watching is almost never caused by the average speed. It is caused by the variation, by the absence of a defined behaviour when nothing arrives, and by data that is acted on after its moment has passed. A robot receiving steady advice at a modest pace behaves beautifully. The same robot receiving mostly quick advice with an occasional long gap behaves badly, because the machine was never told what to do during the gap. Chasing speed can therefore make things worse: an optimisation that improves the typical case while leaving one rare slow path untouched buys a robot that fails less often and just as badly. Fix the shape first — protect the loop, timestamp the goals, define the silence — and then, if the machine still needs more advice than the model can give, work on speed with a clear conscience.

How do you decide what to change first?

Run one experiment before changing anything: disconnect the model, give the robot a fixed goal, and watch the machine execute it. If the motion is smooth and steady with the model gone, your controller is fine and your problem is arrangement, so split the model out of the timed path and add the staleness rule before you consider anything else. If the motion is already rough with no model attached, stop reading about models entirely, because the trouble is in the loop, the driver or the hardware, and adding intelligence on top of a shaky foundation only makes the shaking harder to attribute. Two further checks resolve most of what remains. Ask what the robot should do during a long silence and write that behaviour down as a rule rather than leaving it as whatever the code happens to do. Then ask whether a stale instruction could still be executed anywhere in your system, and close that path, because that single case is the one that eventually drives a confident machine into something that moved.

Decide by situation rather than by instinct:

• If the robot is smooth without the model and rough with it -> split the model out of the timed path, because the arrangement is the fault and no purchase fixes arrangement.
• If the robot is rough with no model attached -> fix the loop, the driver or the wiring first, because intelligence on a shaky foundation is unattributable.
• If your robot navigates or manipulates and spans machines -> ROS 2, because goals, refusals and feedback are already modelled there.
• If a model process and a control loop share one board and copy large data -> a shared-memory middleware, because the copying is what steals the time.
• If the arrangement is right and advice still arrives too rarely -> buy the accelerator now, because at this point the money buys something you can measure.

When the foundation question becomes real, weigh candidates on the five axes of the HORUS Fit Framework — ecosystem size, setup effort, team size fit, deployment target and licence — and take the option that loses on the fewest, with no scoring involved. If your robot is one machine running a model beside a control loop, star HORUS on GitHub so it is in your list when you start building, and whether this whole field is where you belong is the honest companion question to this one.