SILO: Simulation-In-the-Loop Sim-to-Real Transfer for Multi-Stage Cable Routing

Anonymous

Abstract

Linear-deformable manipulation remains challenging due to the complex deformations of objects such as cables and ropes. Prior data-driven approaches, particularly imitation learning, have shown some promise in narrowly defined settings but typically require thousands of demonstrations for specific tasks and cable types, limiting scalability and generalization. We introduce a sim-to-real reinforcement learning (RL) framework for multi-stage cable routing that leverages GPU-parallelized simulation to approximate linear deformable behaviors. Training across thousands of parallel simulations enables the learned policies to generalize across diverse cable geometries and deformation patterns. To bridge the sim-to-real gap, we propose a novel deployment strategy that combines a Simulation In the Loop (SILO) execution framework, localized RL policies, and robust cable state estimation. On real-world cable routing tasks, our approach achieves higher success rates and ~2× reduction in cycle times compared to prior state-of-the-art learning methods. To our knowledge, this is the first successful sim-to-real transfer of RL policies for multi-stage cable routing.

Results and Visualizations

Acompanying the paper, we provide video visualizations of various aspects of our system in the real-world and simulation.

Example Deployments

Some example zero-shot sim-to-real deployments at 5x speed of our system. Our system can handle different configurations as well as different cable types. Furthermore, the solve speed is ~2x faster than the baseline with higher success rates.

State Estimation Visualization

Video below shows how every prediction the state estimation system makes as our system routes a cable through multiple harnesses. Left video shows the simulation digital twin that is synchronized with the real world, with the cable visualized as a sequence of colored points. The right side shows an overlay of the real image, simulation image, and the predicted cable segmentation mask.

Learned behaviors

Videos below show some of the emergent behaviors we observed our RL policy learning. None of these behaviors are explicitly programmed or rewarded, but are learned due to the simple nature of the reward function.

Swinging behavior

The two videos below show the emergent swinging behavior learned by the RL policy. The robot executes lateral movements in response to the curvature of the cable relative to the harness. Notably, this behavior is not explicitly encoded in the reward function and emerges from interaction with the simulated dynamics during training.

When the cable is curved to the right of the harness, the policy swings the gripper leftward around the fixture.

When the cable is curved to the left of the harness, the policy swings the gripper rightward around the fixture.

Angling behavior

The video below is an overlay of two videos. The normal-color (white rope and gripper) overlay shows the RL policy changing the gripper orientation as it routes the cable. The blue-tinted overlay shows a scripted baseline that follows the same translational trajectory as the RL policy but maintains a fixed gripper orientation. The video demonstrates how the RL policy's learned behavior depresses the cable further down towards the harness, which helps lead to more successes and faster routes.

SILO vs Sim-to-Real/System ID Tuned Controller

Below compares our SILO method against a sim-to-real/system ID tuned controller for reducing sim-to-real dynamics gaps. Overall we find in our ablation study environment that SILO achieves 15/20 successes on the routing task, whereas traditional sim-to-real deployment with system ID achieves 10/20 successes. The difference is fairly subtle, but given similar starting conditions as shown in the videos, SILO's minimal controller dynamics gap allows for successful RL skill execution in routing, whereas the sim-to-real tuned controller just fails. While SILO achieves higher success rates, it's deployment speed is slower due to the start-stop motions used to help sync with the digital twin.

Sim-to-real tuned controller

SILO

SILO Obstacle Avoidance Example

Below we demonstrate how SILO enables implicit obstacle avoidance during deployment. The video is two videos overlaid, with one video using SILO with the harness modelled in the simulator, and the other does not model the harness. Since in SILO we take actions in simulation and command the real robot to move to the simulated joint angles, we guarantee that the real robot will not collide with the harness if we model the harness in the simulator.

Cable Properties in Real and Simulation

Below shows videos of the cables tested in the real-world as well as in simulation with varying parameters. While the simulated cables are not modelling the real cables 1:1, we find with sufficient randomization during RL training in simulation states achieves a wide coverage of possible cable states that leads to successful sim-to-real transfer of simulation trained RL policies.