I Know Python and LLMs — How Do I Get Into Robotics?

Start on ROS 2 with a simulated robot rather than a leaner stack such as HORUS or a hand-rolled build, because the ecosystem carries you. Almost everything you know about Python, models and serving transfers; what is new is timing, coordinate frames and hardware that lies. That flips only when you already have one machine, a deadline and a control loop that stutters. The rest of this post is for someone shipping Python and language-model work today who wants a route into robots without a degree.

You can get a model to produce a sensible plan for tidying a room, and you cannot get a wheeled thing to cross that room without clipping a chair leg. The gap is disorienting, because in your day job the hard part is the model and everything around it is plumbing you already know how to write. Robotics inverts that. The model is often the easy part, sometimes a small part, and the plumbing has strong opinions about clocks.

Maybe you already ran a robot tutorial, watched a simulated arm wave, and could not tell whether you had learned something transferable or simply typed what the page said. Maybe a kit is sitting in a box because the setup instructions assumed a Linux version you do not run and a build system you have never met. Job adverts do not help: they list C++, mapping, kinematics, sensor fusion and five years of each, which reads like a wall rather than a door. And every time you ask where to begin, somebody names a different framework and defends it warmly.

The question underneath all of that is narrower. What do I build, in what order, so that in a few months I have a machine that moves and proves I can do this?

What should I learn first if I already know Python and language models?

Learn ROS 2 in a simulator, keep writing Python, and make one simulated robot drive to a spot and stop without you touching it. That is the whole first milestone, and it is deliberately unglamorous. Choose ROS 2 not because it is the most elegant thing available but because every tutorial, every driver and every forum answer assumes it, so starting elsewhere means translating every piece of help you find. Keep Python because your fluency there is real capital: the behaviour layer, the glue and the scripts that talk to a model are ordinarily written in Python on shipping robots too. What you are actually buying with this first project is vocabulary — topics, frames, transforms, timestamps, launch files — and vocabulary is what makes the rest of the field searchable. The pattern to avoid is reading about four frameworks before touching one. It feels like diligence. It produces an engineer who has installed a great deal and finished nothing, which changes nothing about what you can claim or apply for.

What is robotics middleware, in plain terms?

Middleware is the postal system that lets the separate programs inside a robot talk without knowing about each other. A camera driver publishes pictures. A perception program reads pictures and publishes something like "there is a cup at the left edge of the table". A planner reads that and publishes a target. A controller reads the target and publishes motor commands. None of those programs holds a reference to any other; each publishes to named channels and subscribes to named channels, and the middleware handles delivery, discovery and the awkward question of what happens when one side is slower than the other.

This matters more than it sounds like it should, for two reasons. The first is that it decides how you debug. You can tap any channel and watch what is flowing through it, which turns out to be the single most useful habit in robotics. The second is that it decides your timing. Every hop through the postal system costs something, and when messages have to be packed up, copied and unpacked on the way between processes, that cost shows up as a robot which hesitates rather than as an error in a log.

What are my actual options for the software layer?

Six options exist for someone arriving from Python and models, and they suit different starting points rather than competing head to head. ROS 2 is the default: the largest ecosystem, navigation and mapping packages you would not want to write yourself, drivers for most hardware you can buy, and a learning curve that is real. HORUS sits in the same slot with the opposite trade — an open-source Apache-2.0 middleware for Rust, Python and C++ where all three languages share the same shared-memory ring buffers, so a Python behaviour script and a compiled control loop hand data to each other on one machine without a packing step in between, which is narrow and genuinely useful once a particular robot starts hesitating. A vendor SDK is the quickest route when a specific arm or base already sits on your desk. A learning-robot policy stack suits people whose real interest is manipulation. MQTT or sockets you write yourself suit small telemetry projects and stop being pleasant the moment a third program appears. Firmware alone suits people drawn to the metal.

How do the options compare for someone coming from Python?

Read this table by what you already have, not by what you eventually want, because most of these routes end in the same career and only some of them work for you this month. Two warnings before the rows. None of them are exclusive — the ordinary professional path is the ecosystem first and a leaner transport later, when a particular machine demands it, which is far easier than the reverse. And no row substitutes for finishing something: a route is only good if it ends in a machine that moved.

Which parts of my LLM skill set actually matter on a robot?

More of it than you expect, and not the parts people brag about. Prompt design matters least. The durable skills are the ones you built around models rather than inside them: you already think about what to do when a call takes longer than the caller can wait, about queues that fill, about retries that make things worse, about a system whose parts fail independently and lie to each other while doing it. A robot is that, with a body attached and a harder deadline. Your instinct to log everything and replay it later becomes the recording-and-playback habit that professional robotics runs on. Your comfort with Python means the behaviour layer belongs to you from day one.

What does not transfer is the assumption that a late answer is still an answer. In a chat product, a slow reply is a worse experience. On a robot, a late reply means the wheel command arrives after the wheel needed it, and the machine wobbles, overshoots or stops dead. Finding where that boundary sits on your particular robot is the main conceptual jump, and it is a jump rather than a wall.

What hardware should I buy to start?

Buy one small wheeled robot with a camera and an onboard computer you can log into over the network, and spend as little as you can while keeping those three properties. Wheels rather than an arm, because a mistake sends the machine into a table leg instead of into itself, and driving exposes nearly every concept you need: odometry, coordinate frames, sensor delay, planning a path, stopping before an obstacle. A real computer on board rather than a microcontroller, because you want to run Python and a model on the machine and watch what that does to everything else sharing it.

Resist buying a humanoid or an expensive arm first. The lesson you need early is not dexterity, it is that hardware lies. The connector that works until the robot turns. The camera that hands you a picture from a moment ago and says nothing about it. The battery that sags and quietly changes the machine's behaviour without your code changing at all. A cheap robot teaches every one of those, and a simulator teaches none of them.

How long does this take with evenings and weekends?

Months rather than years, if the goal is one finished robot rather than mastery of a field. A realistic shape: a few weeks to get comfortable in a simulator and stop fighting the tooling, a few more to get a physical machine driving to a spot on command, then a longer stretch turning that into something which runs unattended with nobody hovering to catch it. Most people describe the same curve — an unpleasant first fortnight where nothing builds and every error message assumes context you do not have, then a fast stretch once the vocabulary lands and the words in the documentation start meaning things.

What stretches the timeline is not difficulty, it is restarting. Every framework switch resets you to the unpleasant fortnight, because concepts transfer between stacks and tooling does not. Set the rule now: no second framework until the first has moved something physical. For a fuller month-by-month version of this route, the software engineer's roadmap into robotics lays out the same order in more detail and applies almost unchanged here.

What if I have never written C++ or Rust?

You can go a long way in Python alone, and you will eventually meet one loop that will not tolerate it. Perception glue, behaviour logic, mission scripts, tooling, data pipelines: Python throughout, on commercial robots, today. The place Python runs out is the innermost control loop, where a pause nobody would notice in a web service turns into a visible twitch in a joint. Most projects handle that by pushing exactly that one loop into a compiled language and leaving the rest of the system alone, rather than rewriting the robot in a new language.

So the honest sequence is: build in Python, find the one loop that misses its slot, then learn enough of one compiled language to move that loop and nothing else. Pick C++ if you want to read the most existing robotics code and match the most job adverts. Pick Rust if you would rather the compiler catch the mistakes that make machines fail at inconvenient moments. The question of whether Python is fast enough for control goes deeper on exactly where that line falls.

What does it look like when a robot project goes wrong?

It looks like a machine that behaves whenever you are watching and misbehaves when you are not. The usual sequence: everything runs beautifully on the desk, the robot goes on the floor, and roughly once a day it clips a doorframe. Nothing crashes. No log shows an error. There is no stack trace to search for, which is exactly why people arriving from software find this so disorienting — the tools that made you effective assume a failure announces itself.

Underneath, it is almost always freshness rather than a bug. A reading was older than the code assumed. Two sources of position disagreed quietly and something averaged them into a confident wrong answer. A queue dropped its oldest message under load while every value on every channel stayed plausible. The habit that prevents most of it is not a package you install: always know how old the data in your hand is, and never treat two measurements as simultaneous merely because they arrived together. That habit is why a robot that works in simulation fails in your kitchen, and it transfers to every stack you will ever touch.

What do I give up by starting on the big ecosystem?

You give up simplicity, some of the machine's resources, and a chunk of your first month. ROS 2 arrives with a build system, a workspace layout, launch files, a configuration language and a set of conventions that exist for good reasons and are not self-evident on day one. On a small single-board computer, the whole apparatus takes a real bite out of memory and processor before a line of your own code runs. And there is a category of problem — messages that vanish on shared Wi-Fi, discovery that finds a laptop it should never have found, tuning knobs whose names mean nothing until one has bitten you — that will eat evenings and teach you little you keep.

Those costs are genuine and still worth paying at the start, because what comes back is every driver, every navigation package, every tutorial and every person able to answer your question at midnight. The trade only turns bad later, on a specific machine, with a specific complaint about timing, which is a far nicer problem to have than a blank workspace and no map of the territory.

When is ROS 2 the better choice?

ROS 2 is the better choice most of the time, and almost certainly for you right now. ROS 2 wins whenever the ecosystem is the point: you need mapping and navigation that already work, a driver for the lidar you just bought, a simulator that speaks the same language as your code, or teammates you can hire who already know the stack. ROS 2 wins for research groups, for robots spread across several computers, and for anyone whose main risk is not finishing. It wins on employability, plainly — the adverts say ROS 2, so learning ROS 2 is career progress in a way that learning a lean transport is not. HORUS is not the answer to any of those problems and does not pretend to be: it replaces the messaging layer, not the ecosystem, so reaching for it early means writing the navigation, the drivers and the tooling yourself, alone, while you are still learning what those words mean. Reach for a lean middleware only with a working robot, a hesitation you can reproduce, and reason to believe the messaging layer causes it.

Can a language model just write the robot code for me?

No, and here is why: the model writes plausible code, and a robot punishes plausibility harder than any other kind of software. Assistants are genuinely useful here — they write boilerplate, explain error messages that assume five years of context you do not have, and get you through configuration files faster than the documentation does. What they cannot do is know things nobody ever wrote down: which direction your particular camera is mounted, why the arm reaches for a point slightly to the left of the cup, or that the sensor you trusted stamps its readings with a different clock from everything else on the machine.

The failure is quiet, which is the dangerous part. Generated code compiles, runs and moves the robot approximately correctly, so the mistake surfaces days later as a machine that occasionally misses. Use assistants for things you could verify yourself, and read the honest answer on chat models controlling robots before trusting one with anything that moves.

Is a robot basically a model with wheels?

Partly, but not the way you think. There is a real sense in which robots are becoming model-shaped: vision-language models propose what to do next, learned policies handle grasps nobody could hand-code, and the field is moving that way quickly enough that the instinct is not wrong. What it misses is the layer underneath, which is not going anywhere. Something has to run the joint controller on a fixed schedule that never slips. Something has to notice the arm is about to hit the table and stop it, on a path that consults no model at all. Something has to keep track of where the robot is during the second the camera is blinded by a window.

The useful mental picture is a slow thinker sitting on top of a fast reflex. Your models live in the slow layer, where a pause is survivable and a wrong answer is recoverable. The reflex layer is small and unglamorous, and it is the part that stops a machine from breaking itself. Learning robotics mostly means learning that second layer, because you already have the first.

How do I decide what to build this month?

Pick the smallest machine-shaped goal you can finish in four weeks and refuse every other robotics activity until it is done. Concretely: a robot that drives to a marked spot and stops with nobody touching it, or a camera and a small arm that pick up one specific object from one specific place. Write down what "done" means before you start, and include the unattended part, because that is where the lessons that change you are hiding.

Then answer one question honestly. Is your problem a learning problem or a machine problem? A learning problem — no robot yet, no deadline, career ambitions — has one answer, and it is the popular ecosystem in a simulator with Python on top. A machine problem — a robot on the bench, a demo date, a control loop that hesitates — is a different question entirely, and the beginner's route to connecting a model to a real robot is a more useful next read. Nearly everyone reading this has the first problem, including the people convinced they have the second.

Take the line that matches you:

• If you are an LLM engineer with no robot yet -> ROS 2 in a simulator with Python on top, because the ecosystem answers your questions while you sleep.
• If you have a cheap wheeled robot on the desk -> the same route, but on the hardware, because physical failures are the ones simulation hides.
• If you already own a specific arm or base -> the vendor SDK first, because a machine moving this weekend beats a tidy architecture next month.
• If you ship a robot already and it hesitates -> a lean middleware under your existing behaviour code, because that is a timing problem, not a learning problem.
• If you keep starting and abandoning -> one robot, one task, four weeks, because the missing ingredient is finishing rather than knowledge.

The HORUS Fit Framework is the checklist behind those lines, and not one of its five axes is a number: ecosystem size, setup effort, team size fit, deployment target and licence. Score any route you are weighing on all five, and the axis you cannot compromise on decides it for you.

Learn the ecosystem, finish one machine, and keep Python where Python belongs. When you later meet the robot that needs a leaner layer underneath, HORUS is open source under Apache-2.0 at github.com/softmata/horus — star it so it is in your list when you start building.