ACE Journal

Sim-to-Real Transfer for Warehouse Manipulators

Who this is for: robotics and reinforcement-learning engineers training manipulation policies in simulation for real warehouse deployment. If you are running grasp or pick-and-place policies in Isaac Sim or MuJoCo and trying to get them onto an actual arm picking actual totes, this is the transfer path and the failure list. You should be comfortable with PyTorch, basic RL or behavior cloning, and the idea that a physics simulator is an approximation you tune, not ground truth.

The arm that works in sim and flails on the dock

The recurring scar in warehouse manipulation looks like this. A grasp policy trains for a week of wall-clock time across thousands of parallel simulated environments, hits a clean success rate in sim, and gets flashed onto the real arm. The first tote of rigid boxes goes fine. Then a half-crushed carton comes down the conveyor, the policy commands a grasp pose that would have worked on the pristine sim mesh, the gripper closes on air or on a corner, and the item tumbles. The policy did exactly what it learned. It is just that “works on the simulator’s idea of a box” and “works on the box that actually arrived” are different questions, and pure sim training only answers the first one.

The naive fix is to train on the real arm. That does not survive contact with a budget. Real manipulation data is slow to collect, expensive to reset, and risky to gather at the volume modern policies need. The economics are the whole reason simulation exists in this stack: you spend compute, not robot-hours, and you get unlimited resets, perfect state labels, and parallelism a physical cell can never match.

Sim-to-real transfer is the discipline of spending that cheap sim budget in a way that survives the move to hardware. It rests on a few deliberate choices: which simulator to run for which objects, which parameters to randomize and on what schedule, how to fine-tune on a small real dataset without losing what sim taught, and how to audit the inevitable failures once the arm is live. Figure 1 shows the full pipeline before the article walks each stage in turn.

flowchart LR
    Sim[simulator<br/>Isaac Sim / MuJoCo] --> DR[domain<br/>randomization]
    DR --> Train[train policy<br/>parallel rollouts]
    Train --> Eval[sim benchmark<br/>eval]
    Eval --> FT[fine-tune on<br/>50-200 real demos]
    FT --> Reg[regression test<br/>vs sim benchmark]
    Reg --> Deploy[deploy +<br/>failure audit]
    Deploy -. confidence below threshold .-> Human[human-in-the-loop]
    Deploy -. logged failures .-> DR

Figure 1. Full sim-to-real pipeline from simulation through deployment; note the dotted feedback edges that route logged failures back into domain randomization and low-confidence picks to a human.

The dotted edges matter as much as the solid ones. Deployment feeds failures back into the randomization ranges for the next training cycle, and a confidence gate routes hard picks to a human instead of dropping them. The pipeline is a loop, not a one-way street.

Simulator choice by object class

Isaac Sim and MuJoCo remain the two most common substrates for manipulation work in 2026, and the choice between them is not about which is “better.” It is about what you are picking.

Isaac Sim runs on NVIDIA’s PhysX backend and its strength is throughput. It executes thousands of environments in parallel on the GPU, so you can collect the enormous rollout volume that RL needs in reasonable wall-clock time. PhysX handles rigid-body contact well enough for well-defined SKUs: boxes, totes, rigid clamshells, anything whose geometry and mass are stable. When your catalog is mostly rigid and your bottleneck is sample count, Isaac Sim is the throughput play.

MuJoCo comes at it from the contact-accuracy end. Its solver is known for stable, well-behaved contact dynamics on small and irregular objects, which is exactly where PhysX needs the most hand-tuning. Soft goods, poly bags, partially torn cardboard, foam packaging, still require significant tuning of compliance and damping in either simulator to resemble reality, but teams grasping small or deformable items tend to report steadier contact behavior in MuJoCo. The cost is speed: MuJoCo’s GPU-parallel story (via MJX) has narrowed the gap but Isaac Sim still wins at raw scale.

The practical rule is to pick by object class rather than by allegiance. Rigid, high-volume SKUs where you need millions of rollouts favor Isaac Sim for throughput. Irregular or deformable items where a wrong contact model silently teaches a bad grasp favor MuJoCo for accuracy. Plenty of teams run both: MuJoCo to validate contact behavior on the hard items, Isaac Sim to mass-train once the contact model is trusted. Figure 2 maps the decision by object class.

flowchart TD
    Q{object class?}
    Q -->|rigid SKUs<br/>boxes, totes| Isaac[Isaac Sim<br/>PhysX, GPU-parallel<br/>throughput]
    Q -->|small / deformable<br/>poly bags, foam| Mujoco[MuJoCo<br/>contact accuracy]
    Isaac --> Both[validate hard items in MuJoCo,<br/>mass-train in Isaac Sim]
    Mujoco --> Both

Figure 2. Simulator selection by object class: rigid, high-volume SKUs favor Isaac Sim for throughput; small or deformable items favor MuJoCo for contact accuracy; many teams use both in combination.

It is worth naming the other substrates so the choice is informed. SAPIEN is a strong option for articulated-object and part-level manipulation, and Gazebo remains common where you want tight ROS integration and full-cell simulation rather than GPU-massive policy rollouts. They are not the default for high-throughput grasp learning, but they fit adjacent problems, and the related-work section returns to how these trade off.

Curriculum domain randomization

Domain randomization is the lever that teaches a policy to ignore the specifics of the simulator and latch onto what transfers. By varying visual and physical parameters across training episodes (lighting, exposure, mass, friction), you force the policy to be robust to the fact that none of those will match reality exactly. If the policy only ever sees one lighting condition, it will overfit to it; vary lighting across a plausible range and it learns a representation that survives the warehouse’s actual lighting.

The parameters that matter most for warehouse picking and illustrative ranges:

# domain_randomization.yaml
# Ranges are illustrative starting points; tune to your cell and catalog.
visual:
  lighting_intensity_lux: [500, 2000]      # dim aisle to bright dock door
  camera_exposure_ms: [5, 20]              # motion blur vs underexposure
  texture_randomize: true                  # swap bin/floor textures per episode
  background_clutter: [0, 8]               # distractor objects in frame

physical:
  object_mass_scale: [0.7, 1.3]            # +/- 30% of nominal SKU mass
  surface_friction: [0.2, 0.8]            # waxed cardboard to rubberized tote
  object_pose_jitter_mm: [0, 15]           # localization / placement noise
  gripper_payload_noise: [0.0, 0.05]       # actuation slop

# curriculum: fraction of the range opened, by training progress
curriculum:
  schedule:
    - { progress: 0.00, range_fraction: 0.10 }   # start narrow
    - { progress: 0.30, range_fraction: 0.35 }
    - { progress: 0.60, range_fraction: 0.70 }
    - { progress: 1.00, range_fraction: 1.00 }   # full width by end

The single most common mistake is randomizing too broadly too early. If you open every range to maximum on episode one, the policy faces a chaotic world before it has learned even a basic grasp, and it converges on the only thing that survives chaos: a conservative, slow, over-cautious grasp that hedges against everything. That policy transfers, but it is sluggish and it fails on tight-tolerance placements where you actually need precision.

The fix is curriculum domain randomization: start with narrow ranges so the policy can learn a competent grasp in an almost-deterministic world, then widen the ranges progressively as training proceeds. The policy first learns to pick, then learns to pick robustly. The widening schedule is the curriculum, as Figure 3 illustrates.

graph LR
    A[start<br/>10% range<br/>learn to grasp] --> B[30% progress<br/>35% range]
    B --> C[60% progress<br/>70% range<br/>add robustness]
    C --> D[end<br/>100% range<br/>full robustness]

Figure 3. Curriculum domain randomization schedule: the randomization range opens from 10% at the start to 100% by the end of training, letting the policy first learn a competent grasp before adding robustness.

Wiring the schedule into a training loop is mechanically simple. The randomizer reads the current range fraction from the curriculum and samples within the opened sub-range:

def sample_randomized_params(cfg, progress):
    """Sample DR params scaled by the curriculum's open fraction at this progress."""
    frac = curriculum_range_fraction(cfg["curriculum"]["schedule"], progress)
    params = {}
    for group in ("visual", "physical"):
        for name, bounds in cfg[group].items():
            if not isinstance(bounds, list) or len(bounds) != 2:
                continue  # skip flags like texture_randomize
            lo, hi = bounds
            center = 0.5 * (lo + hi)
            half = 0.5 * (hi - lo) * frac          # widen with the curriculum
            params[name] = uniform(center - half, center + half)
    return params

The center of each range stays fixed; only the half-width grows with frac. Early episodes sample tight around the nominal value, late episodes sample the full band. That is the entire trick, and it consistently beats both fixed-narrow (does not transfer) and fixed-wide-from-zero (slow, conservative).

Sim-to-real fine-tuning without forgetting

Domain randomization narrows the gap; it rarely closes it. Real friction, real sensor noise, real deformation under a real gripper are not perfectly in your randomized distribution, so a sim-trained policy usually arrives on hardware with a residual gap. The economical way to close it is a small set of real demonstrations, on the order of 50 to 200 episodes, used to fine-tune the policy with behavior cloning or a light touch of online reinforcement learning. That is a few hours of teleoperated or scripted picks, not millions of robot steps.

The trap is catastrophic forgetting. Fine-tune aggressively on a narrow real dataset and the policy will happily overwrite the broad competence it learned across thousands of randomized sim conditions in order to nail the handful of real cases in front of it. It gets better at the demonstrated picks and quietly worse at everything else, and you will not notice until an edge case it used to handle shows up in production.

There are two standard mitigations, and they are not mutually exclusive. The first is to freeze the early layers of the vision backbone, the general-purpose feature extractors that already work, and fine-tune only the later policy layers. The second is elastic weight consolidation (EWC), which adds a penalty that resists changing the weights that mattered most for the original task, estimated from the Fisher information. Freezing is blunt and cheap; EWC is gentler and lets the whole network adapt a little while protecting what matters.

import torch

def freeze_vision_backbone(policy, freeze_until="layer3"):
    """Layer-freeze: stop gradients on early backbone layers, fine-tune the rest."""
    freezing = True
    for name, module in policy.vision_backbone.named_children():
        if freezing:
            for p in module.parameters():
                p.requires_grad_(False)
        if name == freeze_until:
            freezing = False  # everything after this trains
    return policy


class EWC:
    """Elastic weight consolidation: penalize drift from sim-trained weights."""
    def __init__(self, policy, sim_loader, lam=4000.0):
        self.lam = lam
        self.star = {n: p.detach().clone()
                     for n, p in policy.named_parameters() if p.requires_grad}
        self.fisher = self._estimate_fisher(policy, sim_loader)

    def _estimate_fisher(self, policy, loader):
        fisher = {n: torch.zeros_like(p)
                  for n, p in policy.named_parameters() if p.requires_grad}
        policy.eval()
        for batch in loader:
            policy.zero_grad()
            loss = policy.bc_loss(batch)      # log-likelihood of sim actions
            loss.backward()
            for n, p in policy.named_parameters():
                if p.requires_grad and p.grad is not None:
                    fisher[n] += p.grad.detach() ** 2 / len(loader)
        return fisher

    def penalty(self, policy):
        out = 0.0
        for n, p in policy.named_parameters():
            if n in self.fisher:
                out += (self.fisher[n] * (p - self.star[n]) ** 2).sum()
        return self.lam * out


# fine-tune loop on the real demos
ewc = EWC(policy, sim_loader)
for batch in real_demo_loader:           # 50-200 real episodes
    opt.zero_grad()
    loss = policy.bc_loss(batch) + ewc.penalty(policy)
    loss.backward()
    opt.step()

After fine-tuning, regression-test against the original simulation benchmark before you trust the policy. The sim benchmark is the cheap, repeatable test set you already have, covering edge cases that are expensive or dangerous to collect on real hardware. If the fine-tuned policy has slipped on the sim benchmark, you have caught catastrophic forgetting in the lab instead of on the dock. The regression check is the guardrail that makes aggressive fine-tuning safe to attempt.

Failure-mode auditing on deployment

A clean transfer is the start of the operational problem, not the end. A live warehouse is a non-stationary, adversarial environment, and a deployed policy needs an explicit failure-mode audit with instrumentation that catches the failures it will have. The common ones are predictable:

The cheap, high-leverage safety layer is to log the grasp planner’s confidence on every attempt and route low-confidence picks to a human-in-the-loop fallback. Confidence below a threshold means “I am not sure,” and the honest response is to hand it to a person rather than fumble it. Operators can tune the threshold post-deployment without retraining anything.

def execute_pick(observation, planner, threshold=0.65):
    plan = planner.plan_grasp(observation)
    log_metrics(
        confidence=plan.confidence,
        predicted_mass=plan.predicted_mass,
        torque_headroom=plan.torque_headroom,
        lighting_lux=observation.lighting_lux,
    )

    # hard safety interlocks first
    if plan.predicted_mass > plan.payload_limit:
        return Fallback(reason="mass_exceeds_payload")     # torque saturation guard
    if not plan.torque_feasible:
        return Fallback(reason="torque_infeasible")

    # confidence gate: low confidence goes to a human
    if plan.confidence < threshold:
        return Fallback(reason="low_confidence",
                        confidence=plan.confidence)         # human-in-the-loop

    return planner.execute(plan)

The deployment audit is itself a flow with explicit branches, and drawing it makes the fallback paths legible to the operators who own the cell. Figure 4 shows those branches and the feedback edges back into training.

flowchart TD
    Obs[observation] --> Plan[grasp planner]
    Plan --> Mass{mass > payload<br/>or torque infeasible?}
    Mass -->|yes| FB1[fallback:<br/>torque guard]
    Mass -->|no| Conf{confidence<br/>>= threshold?}
    Conf -->|no| FB2[human-in-the-loop]
    Conf -->|yes| Exec[execute grasp]
    Exec --> Result{success?}
    Result -->|slip / drop| Log[log failure +<br/>conditions]
    Result -->|ok| Done[place]
    Log -. feed next DR cycle .-> DR[widen randomization]
    FB2 -. teleop demo .-> Demos[real demo set]

Figure 4. Deployment audit flow with safety interlocks: hard payload and torque checks gate before the confidence threshold, and both logged failures and human recoveries feed back into the next training cycle.

Every logged failure and every teleoperated recovery is training signal for the next cycle: failures inform which randomization ranges to widen, and human recoveries add to the real demo set the fine-tuning stage draws on. The audit is how the pipeline learns from production instead of merely surviving it.

Sim-to-real transfer is a crowded field and the choices have real trade-offs worth stating plainly.

On simulators, Isaac Sim and MuJoCo are the two poles discussed above: throughput versus contact accuracy. SAPIEN is the strong choice when the manipulation is about articulated objects and part-level interaction (drawers, lids, hinged packaging) where its asset pipeline shines. Gazebo earns its place where ROS integration and full-cell simulation matter more than GPU-massive policy rollouts, for example simulating the whole work cell including conveyors and safety zones. None of these is universally best; they occupy different points on the throughput-versus-fidelity-versus-integration triangle.

On closing the gap, domain randomization is one of three broad strategies. System identification goes the other way: instead of randomizing to cover your ignorance of the real parameters, you measure them, calibrating the simulator’s friction, mass, and dynamics to a specific cell so sim matches reality more closely. Domain adaptation is a third route, learning a mapping that aligns sim and real observation distributions, often with adversarial or feature-alignment methods, so the policy sees a “realified” input. These are complementary, not competing: a common production recipe is randomize broadly, system-identify the parameters you can measure cheaply, and adapt the vision features that are hardest to randomize convincingly.

On fine-tuning, EWC and layer freezing are the conservative, well-understood options covered here, but they are not the only ones. Adapter or LoRA-style fine-tuning inserts small trainable modules and leaves the base weights frozen, which sidesteps catastrophic forgetting structurally rather than by penalty, and keeps the original policy recoverable by simply removing the adapter. The trade is that adapters add a little inference overhead and need plumbing into the policy architecture, where freezing is a one-line change and EWC bolts onto an existing loss. For a one-off transfer, freeze or EWC is usually enough; for a fleet that fine-tunes per-site repeatedly, adapters are worth the engineering.

Where it breaks

Sim-to-real is not magic, and several situations need care or are genuinely poor fits.

Deformables and soft goods. Poly bags, loose-fill, partially crushed cartons, and fabric are where every simulator’s contact model is weakest. A grasp that the sim scored as solid can collapse on a real bag whose deformation the simulator never captured. If your catalog is heavy on soft goods, expect the residual gap to be large and budget far more real demos, or accept that some SKU classes need a different manipulation strategy entirely.

Torque saturation beyond the randomized range. The policy is only robust within the distribution it trained on. An item heavier than your mass range, a 0.7 to 1.3 scale will not save you from a 2x outlier, can command a motion the arm cannot execute, and the policy has no concept that it is asking the impossible. The torque interlock in the fallback code is a guard, not a cure; the real fix is randomization ranges that bracket your actual catalog plus a hard payload limit.

Lighting drift across shifts. Vision policies that look solid at one time of day can degrade when the lighting changes: a floodlit night dock, a roll door open to daylight, a failed fixture. Randomizing lighting helps, but drift outside the trained range still pushes the input off-distribution. Monitor lighting as a logged signal (it is in the fallback code for a reason) and treat large excursions as a known risk window.

The sim benchmark does not cover real edge cases. Regression-testing against the sim benchmark only protects you from regressions the benchmark can see. If the real world produces failure modes the benchmark never modeled, a specific crushed-corner geometry, a reflective shrink-wrap that confuses depth, the benchmark passes and the arm fails. The benchmark is necessary, not sufficient; the deployment failure log is what closes that gap over time.

Reward and demo quality. Everything downstream inherits the quality of your reward function and your real demonstrations. A reward that does not penalize slippage will train a policy that tolerates slippage; 50 sloppy teleoperated demos will fine-tune toward sloppy behavior. Garbage in, transferred garbage out. The small real dataset is high-leverage precisely because it is small, so its quality matters more per episode than anything in the sim phase.

Further reading

Start with the Isaac Sim documentation [1] and the MuJoCo documentation [2] for the two main simulators, and the Isaac Lab framework [3] for the GPU-parallel RL training layer most teams build on top of Isaac Sim. The foundational domain-randomization paper is Tobin et al. [4], with Peng et al. [5] extending it to dynamics randomization for transfer. For the curriculum idea applied to randomization, OpenAI’s dexterity work [6] is the canonical large-scale demonstration. On catastrophic forgetting, Kirkpatrick et al. [7] is the EWC paper, and Houlsby et al. [8] introduces the adapter-style fine-tuning the positioning section mentions. For the broader sim-to-real landscape, the survey by Zhao et al. [9] is a good map, and SAPIEN [10] and Gazebo [11] cover the adjacent simulators. James et al. [12] on RCAN is a useful reference for the domain-adaptation route.

Written with AI assistance, editorially reviewed.

References

[1] NVIDIA, “Isaac Sim Documentation.” https://docs.isaacsim.omniverse.nvidia.com/

[2] DeepMind, “MuJoCo Documentation.” https://mujoco.readthedocs.io/

[3] NVIDIA, “Isaac Lab Documentation.” https://isaac-sim.github.io/IsaacLab/

[4] J. Tobin et al., “Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World,” IROS 2017. https://arxiv.org/abs/1703.06907

[5] X. B. Peng et al., “Sim-to-Real Transfer of Robotic Control with Dynamics Randomization,” ICRA 2018. https://arxiv.org/abs/1710.06537

[6] OpenAI et al., “Learning Dexterous In-Hand Manipulation,” IJRR 2020. https://arxiv.org/abs/1808.00177

[7] J. Kirkpatrick et al., “Overcoming Catastrophic Forgetting in Neural Networks,” PNAS 2017. https://arxiv.org/abs/1612.00796

[8] N. Houlsby et al., “Parameter-Efficient Transfer Learning for NLP,” ICML 2019. https://arxiv.org/abs/1902.00751

[9] W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: A Survey,” IEEE SSCI 2020. https://arxiv.org/abs/2009.13303

[10] F. Xiang et al., “SAPIEN: A SimulAted Part-based Interactive ENvironment,” CVPR 2020. https://arxiv.org/abs/2003.08515

[11] Open Source Robotics Foundation, “Gazebo.” https://gazebosim.org/

[12] S. James et al., “Sim-to-Real via Sim-to-Sim: Data-Efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks,” CVPR 2019. https://arxiv.org/abs/1812.07252