MuJoCo vs PyBullet: Which for Manipulation Research?

For manipulation research, start in MuJoCo, and choose PyBullet only when the code, the benchmark or the course you must build on already lives there. MuJoCo handles finger-and-object contact in a way a result can survive, ships arm and hand models somebody already tuned, and costs nothing under a permissive licence. The verdict flips when the thing actually breaking is timing between your own programs, which is a ROS 2 or HORUS question rather than a simulator one. The rest of this post is for a researcher or a small team picking one simulator for a grasping or in-hand project this term.

You have a gripper closing on a block, and the block leaves. Not slides, not tips: leaves, as though flicked by something invisible, then reappears a moment later sitting innocently on the table.

Or the fingers pass through the mug and you lose an afternoon reading about collision margins. Or the arm holds the object beautifully until you scale the object down, at which point the whole scene starts to buzz. Or the policy you trained earns its reward every single episode by wedging the object against the side of the bin, in a way no real object would ever wedge, and you realise the number going up was never about grasping at all.

Meanwhile the deadline is real. There is a workshop paper, or a lab meeting where somebody will ask again why the arm still cannot pick up a fork, or an advisor who wants to see the in-hand rotation working before the term ends.

Underneath the frustration sits a question nobody wants to say out loud: whether the simulator you inherited from a tutorial is the reason the work is stuck, or whether the simulator is fine and something else is. Switching costs a week you do not have. Not switching might cost a quarter.

Should manipulation research start in MuJoCo or PyBullet?

Start in MuJoCo, unless the work you are building on already lives in PyBullet, in which case the cost of moving outweighs what you would gain.

Manipulation is the one corner of robotics where contact is the subject rather than a detail. A wheeled robot mostly succeeds by not touching things. An arm exists to touch things, and every question that makes the field interesting, whether the fingers keep hold when the wrist turns, whether the block slides or tips, whether an in-hand reorientation is possible at all, is a question about what happens where two surfaces meet.

A simulator that resolves contact badly does not hand you a slightly wrong answer. It hands you a policy that has learned to exploit the wrongness, and you discover this on the real arm, in front of people.

MuJoCo was built by people who cared about that specific thing, and it shows in small ways: a gripper closing on an object behaves the way a person expects, models arrive tuned rather than merely converted, and scenes rarely detonate when you change a mass.

PyBullet's strengths are different and genuine. PyBullet installs in a minute, runs on any laptop, and speaks Python the whole way down.

What does a physics simulator actually do for manipulation research?

A simulator does three separate jobs, and manipulation work leans almost entirely on the first one. The first job is prediction: where everything ends up a moment from now, given masses, joints, friction and whatever is pressing against what. The second is imagery: what a camera bolted above the workspace would have seen. The third is impersonation, where the simulator presents itself to your code as joints, encoders and a gripper so your program cannot easily tell the hardware is missing.

For navigation or whole-robot integration, the third job pays the rent. For manipulation, the first job is the whole point, because the moment your research is interesting is the moment two objects are in contact and something has to decide what happens next.

This is why the simulator argument is sharper in manipulation than anywhere else in robotics. Elsewhere, a simulator that is roughly right is enough to find your flipped signs and your missing frames. Here, roughly right is the same as wrong, because the policy will find whatever is roughly wrong and build its whole behaviour on top of it.

What are the real options for simulating a hand or an arm?

There are about six that a serious manipulation project would consider, and most labs end up running two. MuJoCo is the default for contact-heavy work, open source under Apache-2.0, with the MuJoCo Menagerie collection supplying arms and multi-fingered hands that were tuned rather than dumped through a converter. MJX is MuJoCo written for JAX, which runs many copies of a scene at once on a graphics card when you are training a policy rather than watching one. PyBullet is the Python-first option that loads URDF files and asks nothing of your hardware. Isaac Lab is NVIDIA's training stack for teams whose behaviour comes out of a camera as well as a hand. Drake is the choice when the maths of the controller matters more than the picture. Gazebo remains the place to test a whole robot with sensors rather than a hand in isolation.

One distinction saves projects a term: the simulator is not the layer your programs talk through. That layer is ROS 2, a vendor stack, or a middleware such as HORUS, an open-source real-time robotics middleware for Rust, Python and C++ where the three languages share the same shared-memory ring buffers, so messages between programs on one machine are not serialised. That project is Apache-2.0 and validated in simulation, and it simulates nothing, renders nothing and replaces neither option in this comparison.

How do the simulators compare for manipulation work?

Read the last column first. Most manipulation projects are eliminated into a simulator by what their code already depends on, what hardware sits under the desk, and whether the deliverable is a paper, a demo or a product.

No column ranks accuracy, because accuracy is a property of your model rather than of the logo on the simulator.

Which simulator fits a solo researcher, a lab, or a product team?

A solo researcher should pick MuJoCo, a lab should pick whatever its existing code and its students already run, and a product team should pick by what the arm has to do on the day it ships.

For one person, the deciding factor is how much of your time goes into fighting the model rather than answering your question. Starting from a tuned hand model is worth more than any argument about solvers, because the alternative is spending your first month discovering that your fingers are too stiff and your object too light.

For a lab, continuity beats correctness more often than professors admit. If four students share an environment suite, a shared repository and a set of results, moving one student to a different simulator means the results no longer compare, and comparison is most of what a lab produces. Change simulator between projects, not inside one.

For a product team, the question is what happens after the paper. A learned grasp that works only in one simulator is a demo. The arm has to run against real drivers, a real camera and a real safety stop, and that world is described more fully in what happens when a robot leaves simulation.

What computer does each simulator expect you to have?

MuJoCo and PyBullet both run on an ordinary laptop with no graphics card, and MJX and Isaac Lab both expect a real NVIDIA card with plenty of memory.

That split decides more projects than any technical comparison. If you are working on a MacBook in a shared office, MuJoCo and PyBullet are both open to you and the NVIDIA-dependent paths are not. If your group has a machine with a good card that everybody books time on, the calculation changes, but shared machines have their own tax: you stop running experiments casually, and casual experiments are where most understanding comes from.

There is a middle route worth knowing. You can build, debug and watch in plain MuJoCo on your own laptop, then move the same model to MJX on a card when you are ready to train something. The model carries over, the intuition carries over, and you only queue for the shared machine when queueing is worth it.

If the graphics card question is what has stalled you, the honest answer about GPUs for simulation covers what you actually need.

How soon do you need a result you can publish or show?

If you need something in two weeks, use whatever is already installed and already loads your robot, and revisit the choice afterwards.

Simulator migration is not a technical task, it is a modelling task. Converting a robot description takes an afternoon. Making the converted robot behave, so the gripper grips instead of vibrating and the object rests instead of drifting, takes days at best, and it takes days that nobody schedules because the conversion looked easy.

On a horizon of a term, the calculation reverses. Two weeks spent moving to a better-fitting simulator early buys back a month of arguing with contact settings later, and it buys you comparability with the environment suites everyone else publishes against.

The trap is the middle case: a month-long deadline with a simulator that is nearly working. That is where people switch halfway, end up maintaining two setups, and deliver neither. Pick one, write down the reason, and do not revisit it until the deadline has passed. The reason matters because in three months you will not remember whether the choice was considered or accidental.

What do you need to know already to be productive in each one?

PyBullet asks for Python and a URDF file, and MuJoCo asks for Python plus a willingness to learn its own model format and what its contact settings actually control.

That gap is smaller than it sounds and larger than it looks. MuJoCo imports URDF, so getting a robot on screen is quick. Getting the robot to behave means writing MJCF by hand: naming contact pairs, setting friction and softness, deciding which joints are actuated and how. It is a real skill, it takes a week or two to acquire, and afterwards it is the skill that makes your results mean something.

MJX adds JAX on top, which is a genuine second language: array-shaped thinking, no Python loops in the hot path, functions that cannot quietly mutate state. Somebody in the group needs to be comfortable there or the training loop becomes a black box nobody can fix.

Isaac Lab adds NVIDIA's toolchain and scene authoring. Drake adds optimisation and a systems framework. Neither is unreasonable, but neither is something you pick up while also doing the research.

What do people try first, and why does it stop working?

Most people start with whatever their course used, tune friction until the block stops sliding, and get further than they expected before hitting a wall.

The wall arrives in a recognisable order. First the object slips, so friction goes up. Then the fingers sink into the object, so stiffness goes up. Then the scene buzzes, so the timestep goes down and every experiment starts taking an afternoon. Then the policy trains beautifully and does something absurd, like pinning the object against the bin wall or exploiting a gap where two contact surfaces overlap. Each fix was reasonable. Together they have produced a world with its own physics, and a policy that is an expert in that world alone.

The tell is that your tuning parameters no longer have physical meanings you can defend. When you cannot say why the friction value is what it is, other than that the block stopped sliding, the model has stopped being a model.

That is the moment a better contact solver stops being a preference and becomes the work. It is also the moment PyBullet's advantages, which are all about starting quickly, have already been spent.

What do you give up by choosing MuJoCo?

You give up the URDF-native world, a pile of older baselines written against PyBullet, and the assumption that everything you need is one pip install away.

The URDF point is the practical one. Robot descriptions in the wider ecosystem are URDF, the tooling around them is URDF, and your arm's manufacturer shipped URDF. MuJoCo will read it, but the parts that matter for contact are not in URDF at all, so you will be maintaining a second description in MJCF and keeping the two honest with each other.

You also give up some reproduction convenience. If the result you want to beat was published with PyBullet environments, matching it in MuJoCo means porting the task definition, the reward and the reset logic, and any of those can differ in ways that make your comparison unfair without anybody noticing.

And you give up photorealism. MuJoCo renders well enough for a person to see what is happening and not well enough to train a policy on pixels that must transfer to a real camera. That job belongs to Isaac Lab or a rendering pipeline bolted alongside.

When is ROS 2 the better choice?

ROS 2 is the better choice the moment the arm is real and other people have to use it. Choose ROS 2 when you need a driver for the actual manipulator, a motion planner, calibration between camera and gripper, coordinate frames maintained for you, and a recording of what every program saw when the run went wrong.

Choose ROS 2 when the next student has to inherit the setup, when a collaborator at another institution should be able to run your code, or when a manufacturer ships an interface and that interface speaks ROS 2. Choose ROS 2 when the arm shares a robot with a base, a lift and a perception stack that someone else wrote, because that integration is exactly what ROS 2 exists to do and rebuilding it yourself is a year of unfunded work.

HORUS is not the answer to any of that. It provides no planner, no drivers, no calibration tools and no ecosystem of existing packages, and a manipulation project that needs those needs ROS 2 instead. The narrower case comes later, when several programs on one machine start interfering and the arm hesitates under load.

Is PyBullet finished now that MuJoCo is free?

No, and here is why: a large amount of published, teachable, working code is written against PyBullet, and rewriting working code is a poor use of a research term.

If your task is to reproduce a result, PyBullet is not a compromise, it is the correct instrument, because reproducing a result in a different simulator is a different experiment. If you are teaching, PyBullet installs anywhere, needs no graphics card, and lets a class of twenty people run the same thing on twenty different laptops in the first hour rather than the third week.

If you are sketching, PyBullet is still the fastest path from an idea to something moving on screen, and a sketch that exists beats a better sketch that does not.

What has changed is the default. When MuJoCo required a paid licence, PyBullet was the reasonable starting point for anyone without institutional money, and an entire generation of code accumulated there. That reason disappeared. So PyBullet is no longer where new contact-heavy research naturally starts, which is a different claim from PyBullet being finished.

Will a better simulator make your grasping policy work on a real arm?

Partly, but not the way you think: a better contact model closes the contact gap and leaves every other gap exactly where it was.

The gaps a simulator cannot close are the ordinary ones. The object in your lab is heavier than the number in your model, and its surface is slightly greasy. The camera hands the policy a picture of where the object used to be, so the fingers close on a memory. The gripper's own controller has a mind of its own between the command and the fingers. Programs on the workstation compete for the same processor, so the loop that should react to a slip reacts a beat late and the object is already gone.

You can tell these apart without buying anything. A physics gap repeats: the same object fails the same way every time. A timing gap does not: the same setup works in the morning and fails when the training job is running, better with half your programs switched off. Before spending a term on fidelity, read why simulation-trained robots keep failing in the real world and run the experiment with everything else turned off.

How do you decide which one to open this week?

Write down the single question your next result has to answer, then pick the simulator that answers it with the least model-building. Not the better simulator. The one that answers your question.

If the question involves fingers holding, slipping, rolling or reorienting an object, contact quality is the experiment and MuJoCo is the instrument. If the question is whether an algorithm learns at all, and contact is scenery, then whatever is installed is fine and switching is procrastination with a technical justification.

Two checks stop most bad decisions. First, look at what your comparison group publishes against, because a result nobody can compare is worth less than a slightly weaker result everyone can. Second, look at what is under your desk, because a simulator only one person in the group can run is a simulator the group does not have.

If neither check decides it, default to MuJoCo, because the cost of starting there and not needing the contact quality is small, while the cost of starting elsewhere and needing it is a term.

• If you are studying grasping, slipping or in-hand motion -> MuJoCo, because contact is your experiment rather than your scenery.
• If you are reproducing or extending published PyBullet work -> PyBullet, because a reproduction in another simulator is a different experiment.
• If you are training a policy over many parallel attempts -> MJX on a graphics card, because volume is the entire method.
• If you are teaching a class on shared laptops -> PyBullet, because everyone running something in hour one beats a better tool in week three.
• If the arm is real and other people must use your setup -> ROS 2, because drivers, planning and calibration already exist there.
• If simulation behaves and the real arm hesitates under load -> stop tuning physics, because the fault is timing rather than contact.

The HORUS Fit Framework reduces the layer underneath to five things you can judge without running a benchmark: ecosystem size, setup effort, team size fit, deployment target and licence. For research groups, setup effort and team size fit usually decide it, and licence matters more than expected on the day a company asks whether your lab's work can be handed over.

The week your simulated grasp works and the real hand hesitates is a bad week to start reading about the layer beneath the simulator. Put HORUS on the shelf before then: star it so it is in your list when you start building.