Physical AI vs World Models: What Each Term Is Claiming

Physical AI and world models are not competing choices: physical AI names a category of product, while a world model names a modelling technique. One describes what a machine is for, the other how a model predicts, so neither tells you how data reaches a motor. That flips when a vendor sells physical AI as a whole stack, because you are then choosing middleware, ROS 2 or HORUS, renamed. The rest of this post is for someone with a working predictive model who now has to make a real machine act on it.

Three announcements land in your feed in one week. A chip company says physical AI. A lab says world model. A keynote says both in the same sentence, and then a recruiter asks which of the two you have experience with. You have trained something that predicts the next frame and picks an action from it, and in the simulator the thing looks close to magic: the block gets picked up, the door swings open, the behaviour looks considered rather than reflexive.

Then the same weights go on a machine and the machine looks unwell. The gripper closes a moment after the object has moved. The arm pauses, lurches, then behaves for a while. Nothing in your logs says anything went wrong, and the same task succeeds twice and fails twice with nothing changed between attempts.

Underneath sits the worry you cannot shake: that the vocabulary is moving faster than your understanding, that somebody has already solved this and given it a name you have not learned, and that you are about to spend a quarter rediscovering a body of knowledge under whichever word won the week.

Do you have to choose between physical AI and world models?

No, because the two phrases answer different questions, and both can be true of the same robot at the same time. Physical AI is a claim about scope: this software is meant to act on matter, in a place, with consequences that cannot be undone by pressing escape. Physical AI says nothing about how the software works inside. A world model is a claim about method: somewhere in the system there is a learned prediction of what happens next, and the system consults that prediction before choosing what to do. A world model says nothing about whether a machine exists. A warehouse robot marketed as physical AI may contain no learned prediction anywhere, running on measured geometry and hand-written rules with a classifier bolted onto the front. A world model may live in a paper, be evaluated on video, and never touch a motor. The two words tell you what somebody is selling and what somebody is researching, in that order. Neither answers the question you actually face, which is what runs on your machine, in which language, and how the pieces reach each other in time.

What does a world model actually claim to do?

A world model claims to predict what the world will look like shortly after an action, so a system can try candidate moves inside its own head before committing to one. That claim matters because it changes what a robot can do with a surprise. A purely reactive policy maps what it sees to what it does, and when a scene is unfamiliar there is nothing to fall back on. A system carrying a learned prediction can roll several candidate actions forward, notice that one of them ends with the cup on the floor, and pick a different one. Two honest limits are worth holding onto. The prediction drifts the further ahead you look, so imagined futures are useful for the next moment and not to be trusted for the next minute. And the prediction knows the world in general, not your machine in particular: nothing in the training data mentioned that your second joint has play in it, or that your camera exposure changes when somebody opens the blinds. Robot foundation models sit in the same family and share both limits.

What are your actual options for putting a predictive model on a machine?

There are about seven realistic options, and most teams move through two or three of them in order. You can write one Python program that reads the camera, runs the model and commands the motors, which is where nearly everyone starts. You can lean on the robot vendor's own kit, usually the quickest route on a complete arm or mobile base. You can adopt ROS 2, the large open ecosystem where drivers, mapping, navigation and visualisation already exist and your model becomes one node among many. You can adopt HORUS, an open-source real-time robotics middleware for Rust, Python and C++ under Apache-2.0, where the three languages share the same shared-memory ring buffers, so a Python process holding the model and a C++ process driving the motors exchange messages on one machine without serialising them. You can keep the model on a workstation and talk to the robot over the network. Or you can build the plumbing yourself. The table below describes situations rather than ranking products.

What fits a researcher who has trained models but never shipped hardware?

Start with one Python program and the vendor's kit, because your first task is finding out which problems you actually have. Someone arriving from modelling work usually holds a sharp map of one half of the system and almost none of the other, and the quickest way to draw the missing half is to make a real machine move badly and watch how it does it. That takes days rather than months, and it converts a vague dread into a list of named problems. What you should not do is adopt a large framework on day one because it resembles what professionals use. Two weeks disappear into build tooling, nothing is learned about the robot, and you quietly conclude that robotics is paperwork. The moment to graduate is specific rather than atmospheric: it arrives when one program can no longer hold both jobs, because the model wants to think for as long as it needs while the motors want an answer on a fixed rhythm. Read what sits between the model and the motor before that day rather than after it.

What should you run on the one computer bolted to the robot?

Keep every process on that computer and stop copying camera frames between them, because copying is what a small board can least afford. A robot computer is usually doing several jobs at once: pulling frames off a camera, running a prediction, holding a control loop steady, and writing a log so you can work out later what happened. When those jobs live in separate processes and every frame is packed up, sent and unpacked again, the board spends a large part of its attention shuffling bytes that never left the machine. That work is invisible in your code and obvious in the motion. The symptoms are ordinary: the arm hesitates when you raise the camera resolution, the fan never stops, and adding one more sensor makes an unrelated part of the robot behave worse. Shared memory addresses exactly this, by letting one process write into a buffer another process reads directly, with no packing step in between. Whether you need it is a question about your machine rather than your model, and the test is whether the motion changes when the volume of data does.

What if the demo has to exist in two months?

Pick whatever requires you to learn nothing new, and be honest that you are building a demo rather than a product. Two months is not enough time to learn an ecosystem and debug a machine at once, so the vendor's kit plus a single Python program is usually right, even knowing the arrangement will not survive contact with the second version. Demos are won by machines that do one thing convincingly on a particular table under particular lighting, and lost by tidy machines that do nothing yet. What matters is being clear about the debt you are taking on. A demo built this way tends to have the prediction, the hardware calls and the safety checks tangled together in one file, sequenced by luck rather than design. That is fine for eight weeks and ruinous for eight months. Decide now which parts you intend to throw away and write those parts carelessly on purpose, while keeping one boundary clean: the line between the model and the commands going to the motors.

What if nobody on the team has written control code before?

Assume the hard part will not be the model, and staff the team around that assumption. A group of strong modelling engineers with no machine experience reliably underestimates the same four things: how much time drivers and calibration eat, how differently a robot behaves on a cold morning, how much of the code turns out to be deciding what to do when a reading is missing, and how long it takes to reproduce a failure that happened once. None of that is intellectually demanding. All of it is slow, and none of it appears in the paper you are implementing. Two moves help more than any framework choice. Put one person in charge of the machine itself, including wiring, mounting and power, because a team where nobody owns the hardware loses days to problems that were never software. And make sure everyone can describe a control loop before anybody argues about which language it belongs in, since the timing of that loop is what most of the later arguments turn out to be about.

What do people try first, and why does it stop working?

Almost everyone starts with one Python process doing everything, and the arrangement stops working on the day two parts of the robot need different rhythms. The single-process design is genuinely right at the start: easy to reason about, easy to restart, and nothing can arrive out of order because nothing runs at the same time. The failure is gradual. The model takes as long as it takes, and while the model is thinking, the code that keeps the wheels straight is not running, so the machine behaves well until the interesting thing happens and then lurches at exactly the wrong moment. The usual next move is to split the model into its own process, which cures the lurch and creates a new problem, because now every camera frame has to travel from one process to the other. Teams then discover that packing and unpacking images is not free, that a queue somewhere is filling, and that the oldest frame in that queue is the one being acted on. That is when the plumbing question stops being theoretical.

What does it look like when the prediction is right and the robot still fails?

It looks like a machine making good decisions at the wrong moments, which is much harder to diagnose than a machine making bad ones. The tell is that your logs look fine. The imagined future was plausible, the chosen action suited the scene it was given, the scene was real, and the gripper still closed on air. What happened is that the frame was already stale when the model saw it, or the command landed after the moment it was meant for. A second signature is inconsistency: the same task succeeds and fails on alternating attempts with nothing changed, because the order of events differs slightly each run. A third is that adding a harmless log line changes the behaviour, which should always alarm you, because it means correctness depends on how long things take. None of these are model problems, and no amount of retraining touches them. Why a policy that works in simulation fails on hardware covers the rest of that gap.

What do you give up by skipping the big ecosystem?

You give up other people's code, other people's tools and the hiring pool, and those three are worth more than newcomers expect. The ecosystem is the product: drivers for hardware you have not bought yet, mapping and navigation representing years of accumulated work, a visualiser that shows what the robot believes about the room, and recording tools that let you replay a bad run at your desk instead of chasing it around a table. Choosing a leaner foundation means doing without those or rebuilding them, and rebuilding a visualiser is a quarter nobody planned for. You also give up shared vocabulary, which matters when hiring and when asking a question in public, because a problem described in common terms gets answered while the same problem described in your own terms does not. And weigh maturity honestly: the leaner options above are younger, several are validated in simulation rather than by a decade of deployed fleets, and a smaller project means fewer people have already hit the bug waiting for you.

When is ROS 2 the better choice?

ROS 2 is the better choice whenever the robot needs software that already exists, which is more often than people arriving from model work expect. A machine that must map a building and drive to a goal is a ROS 2 project, because the navigation and mapping stacks represent work you cannot reproduce alongside your actual research. An arm doing collision-aware planning is the same story. If the depth camera you bought ships one maintained driver and that driver is a ROS 2 package, the decision was made for you at the checkout. If the system spans a robot, a workstation and eventually a fleet, ROS 2 was built for that shape and is well travelled there. And if you sit in a lab where everyone already speaks it, shared vocabulary beats any technical argument, because your colleagues can actually help you. HORUS is not the answer in those situations, and picking it there means rebuilding plumbing that already exists in order to lose the packages that were the reason for the project.

Does a world model remove the need for middleware?

No, and here is why: a prediction about the future still has to be fed by readings that are current and answered by commands that arrive in time. The better the model, the more painful lateness becomes, because a system planning several steps ahead is making a commitment about a world it saw a moment ago, and every extra moment of delay makes that commitment more wrong. Something has to carry frames in and commands out, whether you call that thing ROS 2, a vendor's kit, or four hundred lines you wrote yourself and now maintain forever. The seductive thought is that a model good enough at prediction can absorb the delay by predicting further ahead to compensate. In practice a fixed and known delay can be compensated for reasonably well, and a delay that varies cannot, because the model has no way of knowing which of its imagined futures it is currently living in. Delay on a robot almost always varies, since it depends on what else the computer happened to be doing.

Will larger models make the software layer stop mattering?

Partly, but not the way you think. A larger model genuinely solves one class of problem: it handles scenes the previous model could not, and it needs fewer hand-written special cases propped around it. What a larger model does not touch is the shape of the system. If a camera frame is copied three times on its way in, a better model does not change that, and the design flaw sits there waiting for the next model that is hungrier than this one. If a control loop can be interrupted by a slow logging call, a better model does not stop the interruption; it just supplies something else to blame. And if two parts of the robot disagree about which reading is current, that is a question about how data flows, not about how well anything predicts. The pattern to watch for is a problem that becomes rarer but never disappears after a hardware or model upgrade. Rare-but-present is the signature of a structural fault, and a structural fault comes back on the next machine.

How do you tell which problem you actually have?

Log what the robot decided next to when the command actually landed, and read the two together, because that single comparison separates almost every category of failure. If the decisions are sensible and the commands land later than the moment they suited, you have a delivery problem, and no retraining will help. If the decisions themselves are wrong for the scene, you have a model problem and the plumbing is fine. If both look right and the machine still misbehaves, look at what your code does when a reading is missing, because that path is the least tested part of most robots. And if the answer changes from run to run, the order of events is not under your control, which is a scheduling question rather than either of the above. Do this before choosing a foundation, not after. A team that adopts a large framework to fix a problem they have not named usually finds the problem still there afterwards, wearing unfamiliar tooling. What the phrase physical AI actually covers is worth reading alongside this.

Decide by situation rather than by vocabulary:

• If you are putting a learned policy on hardware for the first time -> one Python program and the vendor's kit, because you need to find out what breaks.
• If the robot must map a building or plan around obstacles -> ROS 2, because that code exists and you will not rewrite it this year.
• If your model is Python and your control loop is compiled, on one computer -> a shared-memory middleware, because the copying between them is where your weeks go.
• If the model is too large for anything on the robot -> a policy server, plus a local behaviour that keeps the machine safe when the link stalls.
• If hardware is still months away -> a simulator and a training loop, and no framework decision at all yet.

When the foundation question does arrive, weigh it on the five axes of the HORUS Fit Framework: ecosystem size, setup effort, team size fit, deployment target, and licence. No scores and no league table, just the option that loses on the fewest axes for the machine in front of you. If your project keeps landing on one computer with a Python model, a compiled control loop, and a growing suspicion that the boundary between them is eating your weeks, star HORUS on GitHub so it is in your list when you start building.