Can You Fine-Tune a Robot the Way You Fine-Tune a Model?

No — fine-tuning improves a policy, but a robot improves through measurement and changes underneath, the layer ROS 2 and HORUS both exist to hold. Weights decide what the arm tries; the rhythm, the limits and what happens when the model is late are settled in ordinary code. That flips when the robot is one supported bench arm and the policy really is the weakest part. The rest of this post is for people who train models, have a robot that has stopped getting better, and are deciding where the next fortnight goes.

The policy works. On held-out episodes it does the task, in simulation it does the task, and on the real arm it does the task often enough that somebody shot a video of it. Then you are asked to make it better, so you do the thing you know how to do: collect more demonstrations, run another fine-tune, watch the curves. The curves move. The arm does not.

So you collect again. Better demonstrations this time, more of them, from a second operator, with more variation in where the block starts. The curves move again. The arm still knocks the block over on some mornings and not others, still overshoots when the room is warm, still fails in a way nobody on the team can describe except as "sometimes".

What is unsettling is that none of your usual instruments help. There is no held-out set for Tuesdays. There is no loss curve for a warm room. You have spent a career learning how to make a number go up, and this machine refuses to expose a number that means very much at all.

Can you improve a robot by fine-tuning it the way you fine-tune a model?

No, because the thing you fine-tune is one component and the thing disappointing you is the whole machine. Fine-tuning a language model is a closed loop you own end to end: data in, weights out, a score on data the model never saw. A robot breaks every link in that chain. The data comes from a machine whose behaviour drifts with temperature, wear and how the cables were routed this morning. The score comes from trials that take minutes each and cannot be run side by side on one arm. And the trained weights are only one of several parts that decide where the gripper actually ends up. You can still fine-tune, and you usually should. The mistake is expecting the ratio of effort to improvement to feel anything like it does on text. On hardware, an afternoon spent making observations arrive when the controller expects them will often move the machine further than another round of training, and nothing in your training tooling will ever tell you that.

What does fine-tuning actually change, and what does it leave untouched?

Fine-tuning changes what the robot decides to try, and nothing else on the machine. Underneath the policy sits a layer that no gradient step touches: drivers that speak your motor controller's private dialect, a loop that runs on a fixed rhythm and closes the gap between commanded and measured joint positions, the bookkeeping that turns a detection in camera pixels into a point in the room, the limits on speed and force, and the rule for what happens when a command does not arrive. The arm stops before it hits the table because code refuses to let it through, not because the weights were shown enough tables. That distinction sounds pedantic until the day a fine-tune makes the policy bolder and the table survives anyway, because the limit held. It is also the distinction that decides where your next two weeks should go. What sits between the model and the motor walks that whole chain from a prediction down to a current in a winding.

What does it look like when a team tries to fine-tune its way out of a machine problem?

It looks like a plateau that no amount of data will move. Three rounds of new demonstrations go in, evaluation on held-out episodes gets better each time, and the physical arm behaves exactly as it did in July. Eventually somebody with hardware instincts asks a question nobody had thought to ask, and the answer is embarrassing. The camera frames the policy is acting on were captured before the last motion finished, so the robot has been steering by a picture of where it used to be. Or the gripper command and the arm command are written by two different programs that were never told to agree on when. Or the loop that moves the joints shares a process with the model, so every prediction stalls the thing that keeps the arm steady. None of those are learning problems, all of them look like learning problems from inside a notebook, and every one of them is invisible in a training curve. What happens when the model is slower than the robot is the most common of the three.

What are your real options for the layer you will end up changing?

You have about seven realistic options, and they differ mainly in how much of the layer beneath the policy you inherit rather than write. ROS 2 with its usual setup is the default for a reason: drivers for most hardware, a control framework, coordinate frames and time handling, recording and visualisation, and a hiring pool that already knows the vocabulary. A vendor SDK with your own Python loop is where nearly every research arm starts and it holds up well until the robot needs something the vendor never imagined. A LeRobot-style research stack covers datasets, training and evaluation, and assumes a machine underneath that already moves when told to. HORUS is an open-source real-time robotics middleware for Rust, Python and C++ where the three languages share the same shared-memory ring buffers, so messages are not serialised between processes on one machine, which fits a Python policy and a Rust or C++ controller sharing one board and passing large observations back and forth; like ROS 2 it is Apache-2.0 and it is validated in simulation. You can also stay in one process, push the inner loop onto a microcontroller, or write the layer yourself.

What should you do if your background is training models rather than building machines?

Draw the machine as boxes and arrows before you touch the training script again. Write down every separate program that runs while the robot is moving, what each one sends, how often it sends it, and which of them would ruin your day by being late. Most people who come from model work have never had to think about a program being late, because in training nothing is late, it is only slow. On a robot those are different failures with different symptoms. Then pick one person on the team, or one contractor, or one friend, who has debugged a motor at two in the morning, and have them read your diagram. They will point at an arrow and say "that one is going to bite you", and they will be right. The second habit worth building early is treating the loop that moves joints as an interface you are forbidden from blocking. I know Python and LLMs, how do I get into robotics covers the rest of the vocabulary gap.

What changes if the robot carries a small computer instead of a workstation?

Everything that was invisible on a workstation turns into a design decision on a small board. On a desktop with spare cores, copying a large image between two programs costs you nothing you would ever notice, so nobody notices. On the computer bolted to the robot, that copy competes directly with the thing keeping the arm steady, and the arm is the one that loses. Heat makes it worse: a board that has been running the model for twenty minutes is not the board you benchmarked cold. This is the point where the way messages move between programs stops being plumbing and starts being a choice with consequences you can see in the machine. It is also the point where large observations passing between a Python process and a control process is worth designing rather than assuming. Best middleware for robots that run an AI model on board goes through that specific trade in more detail than fits here.

What should you do if the demo is in two weeks?

Do not change the stack; make the stack you already have honest about what it is doing. A migration is not a two-week job, and a half-finished migration is worse than the thing you started with. Instead, spend the fortnight on four cheap changes. Record everything, including timestamps for when each observation was captured rather than when it was received, because you will want that recording when the demo goes wrong in front of people. Add hard limits on speed and joint range that the policy cannot talk its way past. Define, explicitly, what the machine does when no command arrives, and test it by killing the Python process on purpose while the arm is extended. Freeze the policy weights the moment the machine behaves acceptably, and resist every urge to squeeze in one more fine-tune. Demos fail on the parts nobody rehearsed, and the parts nobody rehearses are always the ones that only matter when something breaks.

What if nobody on the team has written a control loop before?

Borrow the loop rather than writing one, and spend your learning budget on understanding what the borrowed one is doing. Every serious arm vendor ships a controller, ROS 2 has a control framework with years of argument behind it, and many mobile bases hide the inner loop in firmware where you cannot reach it and should not want to. Writing your own is a genuine skill that takes a while to acquire, and the first version anybody writes has a subtle mistake in how it handles a missed cycle. What you do need to learn, and quickly, is how to read what the existing loop is doing: what rhythm it runs on, what it does when its input goes stale, and what happens when you send it two conflicting commands in the same cycle. That reading is a week of work. Writing your own from nothing is a season, and the season is only worth it when the machine is unusual enough that nothing off the shelf fits.

What do you give up by treating this as an engineering problem instead of a data problem?

You give up the comfortable feeling that progress is a single number going up. Data work has a rhythm that engineers of learned systems find genuinely pleasant: the loop is fast, the feedback is quantitative, and the improvements accumulate somewhere you can point at. Systems work under a robot has none of that. Progress arrives as the absence of a failure that used to happen sometimes, which is difficult to celebrate and almost impossible to put in a paper. You will also split your team's attention, because the people who are good at one of these are frequently bored by the other. And there is a real risk of over-building: spending two months on a beautiful layer for a policy that gets replaced in six weeks by something with different needs. The honest defence against that is to fix only what your recordings prove is broken, and to leave the rest ugly until it earns attention.

When is ROS 2 the better choice?

ROS 2 is the better choice whenever the missing piece is something you would otherwise have to invent from nothing. If your robot drives around, you need mapping, localisation and navigation, and those are years of work that ROS 2 hands you on the first afternoon. If your sensor is unusual, the odds are good that somebody already wrote a driver and argued about it in public. If the robot spans more than one computer, ROS 2's networking is designed for exactly that. If you will hire, ROS 2 vocabulary is what candidates already have. If you collaborate with a university, ROS 2 is the shared language. And if you want to see what the robot believed at the moment it did something strange, the visualisation and recording tools exist today and are used daily by thousands of people. HORUS is not the answer when the thing you are missing is a driver, a planner or a mapper that ROS 2 already ships, and choosing it there means writing that yourself.

Is more data always the cheapest way to make a robot better?

No, and here is why: data buys coverage of situations the policy has not seen, and most plateaued robots are limited by something that is identically wrong in every situation. If your observations are consistently stale by one motion, that staleness is present in every single demonstration you record, so the policy learns to compensate for it rather than being freed from it. If your limit clamps a fraction late, it clamps late in every episode. You cannot demonstrate your way past a constant, because the constant is in the demonstrations too. This is the single most expensive misunderstanding in applied robot learning, and it costs teams months. Data genuinely is the answer when the failures vary with the world: new objects, unfamiliar lighting, a grasp angle nobody thought to record, a task variant the policy has never encountered. The test is simple to describe and uncomfortable to run. If the failure is the same on identical replayed input, more data will not touch it.

Does this mean fine-tuning is pointless on real robots?

Partly, but not the way you think: fine-tuning is not pointless, it is badly placed in most schedules. The sequence that works is to make the machine honest first, then train on it. The sequence most teams follow is the reverse, and it quietly wastes the training. Here is the trap. If you record demonstrations on a rig where observations arrive late and the loop stutters, the policy learns those quirks as if they were physics, and it gets good at compensating for them. Then you fix the rig, which you will, because someone eventually finds the bug. Now your carefully trained policy is compensating for a problem that no longer exists, and it is worse than before the fix. Everything recorded before the fix is contaminated. Fine-tune after the layer under the policy is stable, and fine-tune preferentially on the robot's own failures rather than more of the same successful demonstrations, which teach the policy mostly what it already knows.

How do you tell whether the problem is in the weights or in everything around them?

Replay identical recorded input twice and see whether the robot does the same thing. That one test separates the two worlds better than anything else available to you. If the same observations produce different arm behaviour on two runs, the problem lives in timing, scheduling or the transport between programs, and no training run will touch it. If the behaviour is identical both times and still wrong, the policy is a live suspect. Two more tests are worth the hour they cost. Drive the arm through the exact trajectory by hand, with the policy switched off, and see whether it still misses the object; if it does, the fault is in calibration or the frames, not the weights. And record the moment each observation was captured alongside the moment it was used, then look at the gap over a long run rather than an average of it. What AI engineers misunderstand about robot timing explains why that gap matters more than its typical size.

How should you decide where the next fortnight goes?

Decide by asking whether the failure changes from run to run on identical input, because that single question sorts almost every case correctly. Varying behaviour on identical input means the machine is the suspect, and the fortnight belongs to timing, transport, limits and the control loop. Identical behaviour that is consistently wrong means the policy is the suspect, and the fortnight belongs to data and training. If you cannot run that test because you are not recording enough to replay anything, then your fortnight has already been chosen for you: build the recording first, because every later decision depends on it and none of them can be made honestly without it. Add one more question underneath. Ask what the robot does today when your Python process disappears mid-motion. If nobody on the team knows the answer, find out before you spend two weeks on anything else, because that answer is what stands between a bug and a broken machine. What an AI model cannot decide for your robot lists the rest of those unowned decisions.

Reduced to a decision list:

• If you are seeing the same failure on identical replayed input -> fix the layer under the policy, because no volume of new data changes a constant.
• If you are seeing failures that vary with the object or the light -> collect and fine-tune, because that is real missing coverage.
• If you are on a mobile base that needs mapping and navigation -> ROS 2, because those parts exist and are maintained in public.
• If you are running Python inference beside a faster controller on one board -> the shared-memory row in the table above, because large observations cross without being serialised.
• If you are two weeks from a demo -> freeze the policy and add limits, because a stack change is not a fortnight's work.
• If you are alone on a bench arm that already moves -> a vendor SDK and a Python loop, because the ceremony buys you nothing yet.

When you compare the options, the HORUS Fit Framework keeps the argument on the five axes that actually decide it: ecosystem size, setup effort, team size fit, deployment target, and licence. None of those is a number, and all of them are things you can answer honestly about your own team in an afternoon.

If the shared-memory case sounds like the robot you are building, star it so it is in your list when you start building: HORUS is Apache-2.0 at github.com/softmata/horus.