What is this problem
Sim-to-real transfer is the step where a policy or model trained in simulation (or under controlled lab conditions) is deployed onto physical hardware operating in unstructured real environments. It is distinct from simulation itself: a simulator can be arbitrarily sophisticated, but a policy that performs well inside one still has to survive contact with real sensors, real actuators, and real physical variation it was never exactly trained on.
This covers both the classic sim-to-real gap (simulated physics versus real physics) and the closely related lab-to-field gap (a robot that works reliably in a demo cell but not in an unstructured warehouse, home, or job site).
Closing this gap is what determines whether years of simulated training and synthetic data generation actually translate into a robot that works outside the lab.
The bottleneck and pain points
The core pain point is that policies which look strong in simulation or on benchmarks frequently degrade sharply once deployed on real hardware, because simulators still under-model contact dynamics, friction, cable and wire behavior, lighting, sensor noise, and actuator backlash. Closing that residual gap typically requires real-world fine-tuning or on-site adaptation (hours to weeks of teleoperation, data collection, or on-robot reinforcement learning), which is slow and expensive compared to the near-free iteration loop of simulation.
Early field deployments often need heavy human oversight, and intervention rates (how often a remote operator has to step in and correct or take over) are a common practical yardstick that tends to stay high until a system has accumulated substantial real-world exposure. Measuring genuine progress is itself difficult, since a success rate on a curated benchmark or demo task does not reliably predict performance on the long tail of real-world edge cases, and vendors have an incentive to report benchmark wins that do not survive field deployment.
This is arguably the highest-leverage unsolved problem in robotics, because better sensors, better simulators, more diverse training data, and improved domain randomization are all, in effect, indirect attempts to shrink this same transfer gap rather than separate problems. Whoever meaningfully closes it captures value across nearly every downstream robotics application.