How Much Robotics Do You Need to Know to Work on Embodied AI?

Less than a robotics degree, more than a tutorial: learn control loops, coordinate frames, timing, and one middleware, ROS 2 or HORUS. Embodied work fails at the seam where a model's output becomes motion, and that seam is made of timing, units and coordinate frames rather than machine learning. That changes on a large team with robotics engineers already, where depth in models beats breadth in hardware. The rest of this post is for a machine learning engineer deciding what to study before applying for embodied roles or building a robot alone.

You can fine-tune a vision-language model on a Friday, and you have never been asked what a joint limit is. The job adverts you are reading want that model work, then list eight more things underneath it: ROS, kinematics, control theory, C++, real-time systems, sensor fusion, and something called a URDF that returns no useful search results.

You tried anyway. You got a cheap arm, ran the model, printed the outputs, and they looked correct. The arm moved somewhere else entirely, and you spent a weekend unable to say whether the fault was your model, the driver, the units, or the arm.

So you wondered whether to just take a course. Half the internet says a machine learning background is exactly what robotics needs now. The other half says a coordinate frame will humble you within a week. Both halves are right, which is no help at all when you are deciding what to open tonight.

And underneath sits the honest worry: that you will spend six months learning what a robotics graduate learned in one term, and arrive with less of both skills than anybody is hiring for.

How much robotics do you actually need to know to work on embodied AI?

You need four things, and a degree is not one of them: how a control loop works, what a coordinate frame is, why late data is worse than imperfect data, and one middleware learned properly rather than skimmed. That is a few months of deliberate study for someone who already writes Python.

Those four are where model people actually get stuck. Every robot fault a newcomer meets is one of them wearing a disguise, and none of them appear in machine learning work, so there is no intuition to transfer.

What you can safely skip is longer than what you need. You do not need to derive dynamics by hand, prove a controller stable, design a circuit board, or memorise the mathematics of inverse kinematics, because libraries do all of it. You need only recognise the names so a conversation does not stall.

The uncomfortable part is that the four things you do need cannot be read about. They are learned by making a physical thing move badly and finding out why, which is slower than any course promises.

What is embodied AI in plain terms?

Embodied AI means a model whose output becomes physical motion, and whose input arrives from sensors bolted to something that can break. That single difference from software work reshapes everything around the model.

Three consequences follow. The first is that there is no undo. A wrong answer in a chat window is embarrassing; a wrong answer in an arm is a dent in a table or a broken finger, so the system around the model has to catch mistakes rather than merely log them. The second is that time is part of correctness. A decision made after the moment has passed is wrong even if it would have been perfect a moment earlier, which is a strange idea for anyone whose previous work was judged only on the answer. The third is that data is expensive. Nobody scrapes a billion examples of a gripper closing on a wet mug.

That is why an embodied team looks different from a modelling team. Most of the people are working on the parts around the model, and the model is a component in something larger rather than the product itself.

What are the actual routes into embodied AI work?

About eight routes exist, and most successful transitions combine two of them rather than picking one. Learning ROS 2 properly is the most employable single choice, because it is the ecosystem most robotics companies run and the one adverts name. Learning a simulator plus reinforcement learning suits people who want behaviour to be discovered rather than written, and the separate question of what the robot itself runs is covered in choosing a stack for a reinforcement learning project. Imitation learning and data collection suit people coming from large model work, because the skills transfer almost directly.

Then come the narrower routes. A robot maker's SDK teaches one platform quickly and little that moves elsewhere. Embedded C and microcontrollers take you toward the metal. Classical perception and state estimation teach you to say why sensors disagree. And a single-machine middleware such as HORUS, an open-source real-time robotics middleware for Rust, Python and C++ where the three languages share the same shared-memory ring buffers so messages between programs on one computer are not serialised, teaches how processes divide work. That project is Apache-2.0 and validated in simulation, and teaches nothing about drivers or navigation.

How do the learning routes compare side by side?

Read the last column first, because most people are eliminated into a route by their circumstances rather than choosing one freely. What hardware you can reach, how much time you have before you need income, and whether you will work alone will cross out most of this table.

No column ranks difficulty, because difficulty here is mostly a function of what you already know rather than of the subject.

Which route fits the engineer you already are?

Match the route to the sentence a friend would use to describe your job today, because that sentence contains most of the answer. Four descriptions cover nearly everyone who asks this question.

If you train large models and want to keep doing that, learn imitation learning and data collection, plus enough robotics to debug a demonstration rig. Your existing skills are the scarce ones and you should not abandon them to become an average roboticist.

If you write production software and want to move sideways, learn ROS 2 properly. Your ability to make systems that do not fall over is exactly what robot teams lack, and the ecosystem rewards patience with build systems more than it rewards mathematics.

If you are a researcher chasing new behaviour, learn a simulator and reinforcement learning first and treat the hardware as a later problem. Your output is a result, not a product.

If you intend to build a robot alone, learn the control loop, one middleware and enough electronics to not destroy things. Breadth beats depth when nobody else is on the project.

What hardware do you need before any of this sticks?

You need one physical thing that moves, and it can be cheap and disappointing. A hobby servo arm, a small wheeled base or a single motor with an encoder teaches more in a fortnight than a term of reading, because it produces faults that only exist in the physical world.

Simulation covers a genuine amount before that. Coordinate frames, message passing, control loops, planning and most of ROS 2 can be learned with no hardware at all, and doing so is cheaper and quicker than the alternative. A laptop is enough, and no graphics card is required until you train policies.

What simulation cannot teach is the class of fault that makes robotics feel unfair: the connector that works until the cable bends, the motor that behaves differently when warm, the camera whose frames arrive later under load. Those are the stories interviewers listen for, and they only come from hardware. A sensible first project that produces them is described in what to build first when learning embodied AI, and it should be smaller than your ambition.

How long does each route take before you can do useful work?

Imitation learning is useful within weeks if you already train models, ROS 2 takes a few months to reach genuine comfort, and perception or embedded work takes longer because the foundations are wider. Those are honest ranges for someone studying seriously beside a job.

The number that matters more is time to your first embarrassing hardware failure, because that is when the learning curve actually starts. Everything before it is vocabulary. Aim to get a motor turning under your own code within the first fortnight, however ugly the code is, and let the confusion that follows direct your reading.

The slowest possible approach is the one most people choose: reading broadly for months in order to feel ready. Robotics resists that, because the questions you need answered are not the ones you would think to ask. You cannot know that your units were in degrees and the library wanted radians until something swings the wrong way and you go looking.

Plan for months rather than weeks, and plan for the months to be spent debugging rather than studying.

What level of maths and low-level skill does each route assume?

Less mathematics than the field's reputation suggests, and more systems thinking than anybody warns you about. For most embodied roles the mathematics you truly use is geometry: rotations, transforms between frames, and enough linear algebra to know when a matrix is doing something suspicious.

Perception and state estimation are the exception, and they genuinely want probability and linear algebra you can reason with rather than recite. If sensor fusion is the job, that background is not optional and cannot be bluffed.

The skill that decides more outcomes is systems thinking: knowing which layer a fault lives in, what happens when one program stalls, and why two programs sharing a machine can each look innocent while the robot misbehaves. That habit is why software engineers often outperform mathematically stronger candidates on robot teams.

As for low-level ability, reading C++ is close to mandatory and writing it fluently is not. Python remains the daily language on most embodied teams, and the wider version of this transition is mapped out in moving from Python and language models into robotics.

Where does the gap between model work and robot work actually show up?

The gap shows up in four places, and all four look like a broken model when they are not. Knowing the shapes saves months of blaming the wrong layer.

The first is units and frames. The model outputs a number, the driver expects a different unit, and the arm swings confidently to the wrong place. Everything in the log looks correct because every number individually is correct.

The second is staleness. The image the policy acted on was captured a moment ago, so the gripper closes where the object used to be. The policy is fine and the observation was old.

The third is blocking. One program holds something another needs, and the robot behaves well until a second program starts, then hesitates in a way that never happens on your desk.

The fourth is failure handling. In model work an exception ends a run; in robot work an exception leaves an arm mid-motion under load. The question stops being what went wrong and becomes what the machine should do while it is going wrong.

What do you give up by specialising in embodied AI?

You give up speed of iteration, and that loss is larger than it sounds. A change to a web service is live in minutes; a change to a robot needs hardware that is free, a person to watch it, and a space where a mistake is allowed. Working days become shorter in useful attempts.

You give up a portion of the job market too. Far more employers need someone who can ship a model than need someone who can ship a robot, and embodied roles cluster geographically around the places that build machines. Moving into this field narrows where you can live as well as who will hire you.

You also accept dependence on things you do not control. A supplier changes a component, a driver stops being maintained, a platform is discontinued, and work that had nothing to do with your model quietly becomes your problem.

What you gain is a field where the feedback is honest. The robot either picked the thing up or it did not, and no metric can be argued into a different conclusion.

When is ROS 2 the better choice?

ROS 2 is the better thing to learn for almost anyone whose goal is employment, and that is most people reading this. Job adverts name it, interviewers assume it, and the drivers, navigation stacks, visualisers and answered questions that make a robot project finishable all live in that ecosystem. Learning something else first means arriving with skills nobody has a slot for.

It is also the better choice technically for the robots most companies build. If a machine has to map a building, avoid a chair, plan an arm around its own body or accept new sensors next year, ROS 2 already contains that work and the alternatives do not.

HORUS is not the answer to that question. A middleware that moves messages between programs on one computer teaches nothing about navigation, drivers or the mechanics of a robot fleet, and studying one instead of ROS 2 in order to be hireable would be advice that costs somebody a year. Learn the ecosystem first, then the specialisms.

Do you need an electronics or mechanical background to start?

No, and here is why: on almost every embodied team, the electronics and the mechanics were finished by other people before your work begins. You receive a robot that exists, with connectors that fit and joints that turn, and your contribution is what it does rather than what it is made of.

What you do need is a small, practical layer of respect for the physical parts. Know that motors get warm and behave differently when they do. Know that a connector under repeated bending will fail and that intermittent faults are usually physical. Know that a joint has limits and that commanding past one has consequences. None of that requires a soldering iron.

Where the background genuinely matters is when you are building a robot alone, because then there is nobody to hand you the working hardware, and every mechanical shortcut becomes a software problem later. That is an argument for starting on a robot somebody else designed rather than for learning mechanical engineering, and it is why a cheap commercial arm beats a first custom build.

Will buying a robot arm teach you what employers want?

Partly, but not the way you think. An arm on your desk teaches the lessons that reading cannot: units, frames, timing, and the particular feeling of a machine doing something confident and wrong. That is real and worth the money.

What it does not teach is the part employers are actually buying, which is working on a robot that other people also work on. Nothing about a solo arm exercises code review, logging that somebody else can read, handling a failure at three in the morning, or the discipline of not changing four things at once. Those habits are what separates a hobby project from a candidate.

So use the arm for what it is good at, and then deliberately add the missing half. Contribute to an open-source robotics project, work on somebody else's codebase, or build something with a second person so that decisions have to be explained. A modest project two people finished together is worth more in an interview than an impressive one you built alone, because it is evidence of the thing the job actually is.

How should you decide what to learn first?

Pick the single outcome you want within the next six months, then learn only what stands between you and it. Not the ideal curriculum. The obstacle. Almost every stalled transition into this field comes from studying broadly for a destination that was never named out loud.

• If you want a job at a robotics company -> ROS 2 first, because it is the ecosystem the adverts name and the interviews assume.
• If you already train large models -> imitation learning and data collection, because your existing depth is the scarce part.
• If you want to build a robot alone -> the control loop and one middleware, because breadth beats depth with nobody beside you.
• If sensors disagreeing is the interesting part -> perception and state estimation, because that is a specialism with real demand.
• If everything runs on one computer and stutters -> how processes share a machine, because that is a systems problem rather than a model problem.
• If you have a funded team with roboticists -> stay deep in models, because the team already has the other half.

The HORUS Fit Framework reduces the wider tooling choice to five things anybody can judge without a benchmark: ecosystem size, setup effort, team size fit, deployment target and licence. For someone learning, ecosystem size dominates, because the value of a tool while you are learning is mostly the number of people who have already written down the answer to your next question.

The week you finally get hardware and your model does something confident and wrong is a bad week to start reading about the layer underneath it. Put HORUS on the shelf before then: star it so it is in your list when you start building.