How AI Changed What Robotics Software Has to Do

AI shifted robotics software from writing behaviour by hand to moving data on time; ROS 2 still fits most robots, shared-memory middleware the rest. The behaviour teams once wrote out step by step is now learned, so the work that remains is getting data to a model and its answers to a motor in time. HORUS fits when the model and the control loop share one computer. The rest of this post is for people who train models and now have to choose the software their first real machine will run on.

You train models. Somebody has put a robot in front of you, and the interesting part, the model, is the part that already works. It is everything around the model that is eating your month.

The arm reaches for the mug and closes a fraction after the moment that would have worked. The same setup succeeds, then fails, and nothing in your metrics separates the two runs. Push the machine harder and everything goes soft at once: the motion turns jerky, the board runs hot, and it all reads like a loose cable rather than a bug. You cannot re-run any of it, because the mug was moved by a person who has since gone home.

So you go looking for help, and the answers come back in a vocabulary from another decade. Nodes, transforms, lifecycle states, launch files, message types. Much of it looks like ceremony from an era when robots were programmed one behaviour at a time, and your instinct says a model good enough to drive the machine should make most of it unnecessary. That instinct is half right, which is the worst kind of right, because the wrong half breaks hardware.

How did AI change what robotics software has to do?

The job moved from describing behaviour to delivering data on time. Older robot software was mostly a place to write down what the machine should do: a state machine for the task, a hand-tuned perception pipeline, a script that approached, closed the gripper and lifted. That middle layer is the part a learned policy genuinely displaces, and anyone who has watched a good policy work knows it. What replaces the middle layer is not less software but different software, with a different way of failing. A policy has an appetite for data, wants recent data, and produces answers at a pace it chooses rather than one you set. The layer underneath now has one main duty: keep the model fed with observations that describe the present, and carry its answers to a controller before the machine has moved past the moment those answers were meant to fix. Almost everything else follows from that duty — how many programs you run, which language each one is written in, and what the machine does during the gaps between answers.

What is robotics software, in plain terms?

Robotics software is the layer that lets separate programs on one machine exchange data and keep working when one of them falls behind. A robot is never a single program. A camera driver is one, inference is another, something decides the next action, and a controller commands joints, each running at its own pace and each obliged to carry on when the others are slow. The layer in the middle handles the introductions and the delivery, so the model receives frames without knowing which process produced them, and the controller receives decisions without waiting on the process that made them. That is the whole architectural point: it lets the slow thinking part and the fast reacting part run at different rhythms without one dragging the other down. Every such layer also brings conventions — message shapes, a way to start everything, a way to name coordinate frames — and the surrounding ecosystem of drivers and tools is built on those conventions. A plain-English guide to robotics middleware walks through the idea without assuming any of the vocabulary.

What can you actually build a model-driven robot on?

There are about seven realistic foundations, and they differ mainly in how much existing robot software you inherit against how directly your programs share data. ROS 2 in its default setup is the baseline and the right answer for most machines, because the drivers, the navigation stack, the recording tools and the available people all assume it. ROS 2 with Zenoh underneath keeps that inheritance and copes better when programs are spread across computers or a poor link. HORUS is an open-source real-time robotics middleware for Rust, Python and C++ where the three languages share the same shared-memory ring buffers, so messages are not serialised between processes on one machine, which suits a Python model and a Rust or C++ controller sharing one board and one camera stream; like ROS 2 it is Apache-2.0, and it is validated in simulation. After those come the pragmatic choices: one Python process, a robot learning framework's own tooling, the hardware vendor's SDK, or a small layer you write and maintain yourself.

What should you use if your background is machine learning, not robots?

Start on ROS 2, because the half of this problem you have not met yet is the half ROS 2 has already written down. The gaps are rarely where people from machine learning expect them. Coordinate frames cost a week and feel like an insult to your intelligence. Matching a camera frame to the joint positions at the instant of capture costs another week, and getting it wrong presents as a policy that has quietly degraded since training. Deciding what the machine does when observations stop arriving has no equivalent in offline work at all. Those problems have settled answers in a mature ecosystem and no answers in a layer somebody wrote last month. Adopt one habit on day one regardless of the foundation you choose: never put the model call inside the loop that commands the machine, even when a single program would work today. Splitting them later means rewriting the part of the system you understand least, usually against a deadline, usually while the robot is the only one you have.

What should you use if the model and the motors share one small computer?

On a single small computer, the deciding factor becomes how many copies of your largest messages get made before they reach the thing that acts on them. A small board has a fixed amount of memory bandwidth and a fixed power budget, and shuffling camera frames between programs consumes both while producing nothing a user will ever see. ROS 2 handles this class of hardware perfectly well when the model advises rather than drives, and the ecosystem stays worth more than any saving. The strain shows up when a camera feeds a model, the model feeds something that decides, and that feeds a controller, all on one board, with a copy made at every hop. You experience it as a machine that gets hot, drains its battery sooner than the datasheet suggested, and answers late under load, rather than one that fails cleanly. Middleware for robots that run a model on board covers what these boards tolerate, including the option of moving perception onto a second computer.

What should you use if the robot has to work in six weeks?

Use whatever your team already knows, which usually means ROS 2 or a single Python program. Six weeks cannot absorb a new framework, a new robot and a new deployment target at the same time, and what ruins a short schedule is never that the plumbing was imperfect. It is that the policy never got wired to the arm at all, or that the arm reached for a position the object had already left and swept it onto the floor in front of an audience. Two decisions protect a short schedule whatever you build on. Separate the model and the controller into different programs immediately, so a long inference call cannot freeze the machine mid-motion. Then choose what the robot does when an answer does not arrive: hold, finish the motion already underway, or slow to a stop along a safe path. That choice is what a demo survives on, because an audience forgives a robot that is slow and remembers a robot that lunges. Revisit the foundation afterwards, when you know which part actually hurt.

What should you use if nobody on the team has shipped hardware before?

Choose the option with the most public writing behind it, which today means ROS 2. A team with no hardware history is going to spend its first months on problems that feel beneath them and are not: a cable that reads fine until the arm rotates, a camera that renumbers itself after a reboot, a motor controller that needs a specific start-up order nobody documented. Those evenings go faster when somebody has already been stuck the same way in public. There is a second, less obvious reason. A team new to hardware cannot yet tell a plumbing problem from a mechanical one, and standard tools give you a way to look. Recording a run and replaying it afterwards is how you find out that the wrist was already at its limit before the policy asked for more, rather than arguing about it in a meeting. Write your own layer only once you can say precisely which behaviour of the existing one is unacceptable to your machine, in a sentence that does not include the word cleaner.

What does it look like when the software underneath a model is wrong?

It looks like a machine with a bad connection rather than a machine with a bug, which is why teams lose weeks to it. Learn the signatures. The arm moves in small jerks instead of one continuous reach. The robot acts on where the object used to be and grasps just behind a moving target. It pauses mid-motion for a beat, then carries on as though nothing happened. Under sustained load, every symptom appears at once and the whole thing reads as a hardware fault. Three unrelated causes produce that same set of symptoms. The controller is blocked waiting on a result. Sensor data is copied so many times on the way in that every decision describes a moment already gone. Or the model and the controller are competing for the same processor cores and neither gets a clean run. The distinction matters because the fixes have nothing in common: blocking is fixed by separating loops, copying is fixed by changing how data moves, and competition is fixed by pinning work to cores or moving perception elsewhere.

What do teams try first, and why does it stop working?

Almost everyone starts by calling the model inside the control loop, and that stops working the first time the machine has to move and think at once. The construction is the obvious one: read the sensor, run the model, decide, command the motor, repeat. It works on a bench, it works in a slow demo, and it falls apart as soon as the machine carries momentum, because the world does not wait politely during inference. The second attempt is threads inside one Python program, which helps a little and introduces a class of bug that only shows up under load. The third attempt is separate programs with a queue between them, which is the right shape, and it surfaces a hidden question nobody warned you about: when the model falls behind, do you work through every stale observation in order, or jump to the newest? For a robot acting on the present, newest is nearly always right, and a queue that quietly prefers the oldest gives you a machine that permanently seems to be reacting to the recent past.

What do you give up by leaving the ROS 2 ecosystem?

You give up the largest collection of ready-made robot software in existence, and that loss is usually heavier than anything you gain in how data moves. Sensor drivers go first: somebody has already made your camera, your lidar and your arm work there, and repeating that work produces nothing a user will ever notice. Navigation and mapping represent years of accumulated fixes for situations you have not run into yet. The recording and visualisation tools are how you understand a failure that happened while nobody was watching, which describes most failures on a machine driven by a model. Hiring changes too, because ROS 2 on a CV is a filter you can use and anything smaller is something you teach at your own expense. Community answers count for more than teams admit, since at eleven at night the only useful question is whether anybody has been stuck this way in public before. Leaving is defensible when the machine demands it, and it costs you in those ways rather than in any way a benchmark would reveal.

When is ROS 2 the better choice?

ROS 2 is the better choice for most robots with a model on board, and that is not a polite hedge. If your model advises rather than drives — labelling objects, reading a sign, choosing a destination while a conventional controller handles motion — the ecosystem outweighs every other consideration on the table. If the machine has to map a space or navigate around obstacles, those stacks exist there and nowhere else in comparable condition. If your programs are spread across several computers, a transport built for that spread is the correct structure, and shared memory does nothing for you across a network. If somebody else maintains this after you leave, ROS 2 is the vocabulary they already speak. HORUS is not the answer for a team that needs navigation off the shelf, drivers for unusual sensors, or a stack the next three hires will recognise on sight. The narrow case is specific: one computer, large sensor data, and a model and a controller that must see the same data without copies passing between them.

Did AI make the traditional robotics stack obsolete?

No, and here is why: a policy replaced the middle of the stack, not the bottom. Drivers still have to speak to a motor controller in the manufacturer's own dialect. Something still has to run on a steady rhythm and turn a target pose into joint commands that a machine can actually take. Limits on speed, force and joint range still have to be enforced by ordinary code that cannot be talked out of them, because a policy has no concept of a table edge or a person stepping in. Something has to decide what the machine does when nothing usable arrives for a while, and something has to record the run so you can explain why the arm swung wide on the fourth attempt. None of that is learned, and all of it is what the word traditional was pointing at. Whether vision-language-action models are replacing classic robot software takes the same question from the model's side and lands in the same place.

Is the model the thing in charge of the robot?

Partly, but not the way you think. On nearly every working robot, the learned part chooses what to do and a conventional controller decides how to move, and that controller runs continuously whether or not a new decision has arrived. That division is what makes a machine look composed rather than twitchy: the joints are commanded on a steady rhythm by something simple and predictable, while the model contributes goals at whatever pace it manages. Demos that hand a model direct control of joint commands do exist, and they tend to work in one room under one lighting condition. The consequence for your architecture is that "the model is driving" describes an intent, not a wiring diagram. Somewhere below your policy there is a loop you did not train, holding the arm steady, refusing commands that would exceed a limit, and deciding what happens when nothing new arrives. What changes when a model is driving the robot is a longer tour of that division.

How do you choose without trying all of them?

Answer one question first: does everything on this robot fit on a single computer, and is the data between programs large? That question separates the whole field, and almost nothing else does. If the answer is no on either half — programs on separate machines, or only small messages like poses and velocities passing around — the ecosystem argument wins outright and you should be on ROS 2 without agonising further. If the answer is yes on both halves, and the machine is heavy or fast enough that a late command is a physical event rather than an aesthetic one, then how programs share data is worth deliberating over. The second question is who maintains this in a year. A foundation that only one person on the team understands is a risk that comes due exactly when that person is on holiday. Everything else — the language you prefer, the elegance of the API, what a benchmark suggested — matters far less than teams expect once real hardware is involved.

Decide by situation rather than by preference:

• If your model advises and a conventional controller drives -> ROS 2, because the ecosystem is worth more than anything else on offer.
• If perception and control run on separate computers -> ROS 2 with a transport built for that, because shared memory stops at the edge of one machine.
• If a Python model and a faster controller share one board and large sensor data -> a shared-memory middleware, because copies are what you are losing.
• If the demo is in six weeks -> whatever your team already knows, with the model and the controller split into separate programs from day one.
• If the robot misbehaves and nobody can say why -> record a run before changing any foundation, because the cause is often not the plumbing.

When two options stay close, weigh them on the five axes of the HORUS Fit Framework: ecosystem size, setup effort, team size fit, deployment target, and licence. Take whichever loses on fewest: five plain questions about your situation, not about the software. And if your machine is heading towards a model and a control loop on one on-board computer, star HORUS on GitHub so it is in your list when you start building.