How Do You Program a Humanoid Robot? A Beginner's Map

Program a humanoid on the robot maker's own SDK first, add ROS 2 around it, and only then consider training a policy. The walking and balancing already work on any humanoid you can buy, so your job is everything above them: perception, decisions and safety. That order flips if you build the legs yourself, where control and the layer underneath it, ROS 2 or HORUS, come first. The rest of this post is for someone who has watched the demo videos and wants to know what the first six months actually contain.

You watched a humanoid open a door, and then you watched it fold laundry, and then you watched a person shove it and it stumbled and stayed upright. Somewhere in the comments a person said it was all teleoperated, and somewhere else a person said it was learned end to end, and neither of them showed their working.

So what you want to know is what you would actually type. Not the philosophy, not the funding rounds, not the arguments about whether this is the right shape for a machine. The file you open on a Tuesday, and what goes in it.

The material available is unhelpful in a specific way. Some of it is research writing that assumes you already have a doctorate and a cluster of graphics cards. The rest is a vendor tutorial that gets a robot waving in ten lines and then stops exactly where the interesting part begins. Between the two there is almost nothing that says: here is the shape of the software, here is which part is already finished for you, here is the part you will be writing for a year.

And underneath all of it is a worry you have not said out loud, which is that this might simply be too hard to get into.

How do you actually start programming a humanoid robot?

You start on top of the robot maker's own software, because the hard part underneath is already finished and you are not going to beat it in a year. Every humanoid you can buy arrives with a balance controller and a walking controller already running, maintained by people who have spent careers on that one problem. Your first program does not command joints. It asks the robot to stand, then to take a step, then to move a hand to a point in space.

From there the work grows upward rather than downward. You add a camera and a way of asking what is in front of the robot. You add a task layer that turns a goal into a sequence of motions with checks between them. You add the safety logic that stops all of it when a person walks into the room.

The order matters because each layer needs the one below it to be boring. Nobody debugs a grasping strategy on a robot that falls over, and what a humanoid runs onboard is mostly software you did not write and should not want to.

What does programming a humanoid actually mean in plain terms?

It means writing the part that decides what the robot should do, because the part that keeps it upright runs whether you write anything or not. A humanoid has three tiers of software and they are almost separate products. The bottom tier talks to the motors and keeps the machine balanced, running constantly and quietly, and it comes from the manufacturer. The middle tier turns a request such as put the hand there into a coordinated movement of every joint, respecting limits and the fact that reaching forward shifts the robot's weight. The top tier decides what to want in the first place.

Beginners assume the job is the bottom tier, and it almost never is. Programming a humanoid, for the large majority of people who do it, means perception, task sequencing, error handling and safety, written in Python or C++ against an interface the vendor documented.

That is less glamorous than the videos suggest and considerably more achievable than they suggest too, and it is the reason the field can be entered at all by someone who is not a control theorist.

What are the real options for the software layer you build on?

There are about seven, and most humanoid projects use three of them together rather than choosing one. The robot maker's SDK is where everyone begins, because it is the only thing that talks to the joints. ROS 2 goes on top through a bridge and brings cameras, mapping, planning and the tools for seeing what happened afterwards. A policy-learning stack such as MuJoCo or Isaac Lab is where new locomotion and dexterous manipulation get developed. Plain Python scripts over the SDK are how most demonstrations are actually made. A language model can sit at the very top and turn a spoken request into a plan.

Underneath sits a quieter question about how your own programs talk to each other on the robot's computer, which is ROS 2 for most people and sometimes a middleware such as HORUS, an open-source real-time robotics middleware for Rust, Python and C++ where the three languages share the same shared-memory ring buffers, so messages between programs on one machine are not serialised. That project is Apache-2.0, validated in simulation, and it is not a humanoid SDK, not a controller and not a full replacement for ROS 2.

How do the humanoid software options compare?

Read the last column first, because these are eliminated rather than chosen. The robot you can get your hands on, the language your team already writes and the date somebody expects a demonstration will cross out most of this table before taste gets a say.

Nothing in that table is ranked on capability, because at the beginning capability is not what fails. What fails is a stack nobody on the team can debug on the evening it misbehaves.

Are you a hobbyist, a student, or building a product?

A hobbyist should work in simulation, a student should follow whatever platform the lab already owns, and a product team should choose a humanoid with a documented interface and then stop shopping.

For a hobbyist the arithmetic is brutal and simple: a full-size humanoid costs more than a car, and a simulated one costs nothing. The task-layer skills transfer completely, so a hobbyist who builds a working pick-and-place behaviour in simulation has learned the same thing a lab intern learns, minus the fear of breaking something irreplaceable.

A student should optimise for the platform with people around it. Being the only person in the building who uses a particular robot means every problem is yours alone, and humanoids generate a lot of problems.

A product team has a different trap, which is spending two quarters evaluating robots. The interface matters more than the specification sheet, because your code will live against the interface for years and against the specification only in a slide. Pick the one whose documentation you can read on a Sunday and whose support answers email.

Which humanoid hardware will you actually run this on?

The robot you can get access to decides most of your stack for you, and pretending otherwise wastes a term. A research humanoid from a major manufacturer arrives with its own SDK, its own network setup, its own battery rules, and a bridge into ROS 2 that ranges from complete to hopeful depending on the model and the month.

A smaller educational humanoid, the kind with hobby servos and a plastic frame, teaches sequencing and inverse kinematics honestly and teaches nothing about balance under load, because it does not carry any. That is a fine trade for learning and a poor one for planning a product.

Simulation-only is a real answer and not a consolation prize. A simulated humanoid gives you the same task interface, the same coordinate frames and the same debugging habits, and it is where most of the walking research happens anyway.

If a specific robot is already on the table, the useful question is what the vendor's bridge actually covers, which is the whole subject of what works out of the box on a Unitree G1.

How long before you have something worth showing?

Days for a robot that stands, walks and waves, and far longer than anyone expects for a robot that does something useful without a person watching. That gap is the single most misleading thing about the field, because the first part is genuinely easy now and the second part is genuinely hard, and the videos only show the ends of both.

The first week is a good week. You run the vendor's examples, the robot moves, and you feel the project is going to be fine. The second month is when you discover that reaching for a cup requires knowing where the cup is, in a frame the arm understands, with a confidence you can act on, updated more often than the robot moves.

Then comes the part nobody films: making it work when the light changes, when the cup is a different cup, when somebody stands in the way, when the robot has been running for hours. That is where the months go, and budgeting for it early is the difference between a project that ships and a project that stalls after the prototype.

What do you need to already know before you start?

Python, Linux, and enough geometry to reason about where things are relative to each other. Everything else can be learned in the order the project demands it, which is a better order than any curriculum.

The geometry is the part people underestimate. A humanoid has a frame for the head, a frame for each hand, a frame for the base and a frame for the world, and most beginner bugs are one of those confused for another. The symptom is an arm reaching confidently towards nothing at all. Learning to think in frames and transforms pays back within days.

C++ becomes necessary later, when you touch anything near the control loop, and not before. Reinforcement learning is needed only if you intend to create new motion rather than command existing motion, which is a research career rather than a first project.

If the language question is what is blocking you, it is answered directly in which language to learn first for robotics, and the short version is that you will end up reading both.

What do you give up by building on a robot maker's stack?

You give up control of the layer that matters most when things go wrong, and you inherit somebody else's release schedule. The balance controller is usually closed, so when the robot behaves oddly on a slope you can change parameters and file a support ticket, and that is the whole of your influence.

You give up portability too. The motion calls, the coordinate conventions, the safety model and the way errors are reported are all specific to that manufacturer, and moving to a different humanoid means rewriting the layer that touches the robot. The mitigation is old and boring: keep the code that talks to the robot in one place, and keep the code that decides things ignorant of which robot it is talking to.

And you give up some honesty about what your software can do. Demonstrations built on a vendor's motion library can look far more autonomous than they are, which is fine until somebody asks the system to do the same thing in a different room.

When is ROS 2 the better choice?

ROS 2 is the better choice for nearly every humanoid project that has to perceive a room and act in it, and HORUS is not the answer for a team whose robot cannot yet see a table. The moment your humanoid needs a map, a person detector, a camera calibration, a coordinate transform library or a way to record a run and replay it at a desk, ROS 2 already has that, tested by people with the same cameras.

ROS 2 also wins on the fact that other people can help you. A humanoid produces strange behaviour at inconvenient hours, and having a vocabulary that thousands of roboticists share turns a private mystery into a search result.

And it wins on hiring and handover. A student joining next term already knows what a topic is and how to look inside one. That shared foundation is worth more on a humanoid than on almost any other robot, because there are so many moving parts to explain.

Can a language model program the humanoid for you?

No, and here is why: a language model works at the layer where words describe a goal, and a humanoid falls over at the layer where a foot meets a floor. A model can genuinely turn "tidy the table" into a plausible sequence of steps, and that is a real contribution to the top tier of the software. It cannot tell you that the gripper closed on air, that the arm drifted, that the object was heavier than it looked and the robot's balance has shifted.

The code it writes has the same shape as the problem. It will produce something that runs, that looks like the tutorials it learned from, and that is wrong in the ways that only appear on a real robot: a missing check on whether the motion completed, an assumption that a message always arrives, no handling for the case where the arm is already holding something.

Used as a task planner above a checked motion layer, a model earns its place, which is roughly the arrangement described in connecting a language model to a real robot.

Does buying a humanoid mean the walking is already solved?

Partly, but not the way you think: the robot walks well on the surfaces and in the conditions the manufacturer tested, and your building is not one of them. Vendor walking is excellent on flat laboratory floors and gets noticeably less confident on thick carpet, on a threshold strip between two rooms, on a cable, on a wet patch, and on the sort of gentle slope nobody thinks of as a slope.

Payload changes it too. A humanoid carrying a box in front of itself is a different machine from the one that was tuned, because the weight has moved, and the walking controller may or may not have been told. Battery level changes it, temperature changes it, and a joint that has been working all afternoon changes it.

So the walking is solved in the sense that you do not have to write it, and unsolved in the sense that you will spend real time discovering where it stops working and teaching your task layer to avoid those places.

What breaks first when a humanoid leaves the demo and enters a real room?

Perception breaks first, and it breaks in ways that look like the robot being stupid rather than the camera being confused. The demo room had even light, matte objects and nothing moving. The real room has a window that blows out the exposure every time the robot turns, a glass table the depth camera cannot see, a chair that was not there yesterday, and people who move while the robot is deciding.

The second thing to break is the assumption that a request means the same thing twice. Pick up the cup works when there is one cup. In a real room there are three cups, one of them behind a laptop, and the code has no opinion about which one.

Third is the stop. In a demonstration the operator is standing beside the robot with a controller. In a real room somebody has to be able to stop the machine from the doorway, and that has to work when the network is busy and a program has stopped responding.

How do you decide where to start this week?

Pick the smallest task with a beginning and an end, and write only the part of it that nobody has written for you. Not laundry. Something like: walk to the table, look at what is on it, name the objects out loud, and stop. That task exercises locomotion you did not write, perception you did write, and a task layer that has to handle the case where the table is empty.

Then be honest about which problem you actually have. If the robot does the right thing slowly, you have a perception or a planning problem and the answer is better sensing. If the robot does different things on different runs with the same input, you have a timing problem between programs and no amount of better sensing will touch it, which is why a machine can behave differently every run.

And decide when to leave simulation, because a humanoid is expensive to be wrong about, which is the whole question of when to build the real thing.

Here is the decision in five lines.

• If you have no humanoid and no budget -> a simulated humanoid plus a cheap arm, because the task layer is the transferable skill.
• If you have a bought humanoid -> the vendor SDK first and ROS 2 second, because the joints only answer to one of them.
• If the robot must act on spoken requests -> a language model above a checked motion layer, because words are the part it is genuinely good at.
• If the robot needs a new gait or a new grasp -> a policy-learning stack, because that behaviour is trained rather than written.
• If your own programs keep interfering on the robot's computer -> look at the layer they talk through, because the code is probably fine and the arrangement is not.

The HORUS Fit Framework reduces the software choice to five axes you can judge without a benchmark: ecosystem size, setup effort, team size fit, deployment target and licence. On a humanoid, ecosystem size and deployment target usually decide it, and licence matters the first time a demonstration turns into a product.

The evening your humanoid works in the lab and misbehaves in the corridor is a bad evening to start reading about middleware. Put HORUS on the shelf before then: star it so it is in your list when you start building.