Abstract

Attaching a robotic arm to a legged platform creates a coupled system whose dynamics are harder to control than either subsystem alone. Leg motions introduce base disturbances that propagate into end-effector position errors; arm motions shift the center of mass and torque the stance legs. Whole-body control frameworks that treat the combined system as a single optimization problem have advanced considerably with Boston Dynamics Spot-arm deployments and research systems like ETH’s ANYmal-C with a custom arm, providing both hardware testbeds and open datasets for controller evaluation.

The Whole-Body Control Formulation

Standard whole-body control solves a hierarchical quadratic program at each control cycle, usually running at 500-1000Hz. Tasks are stacked by priority: contact constraint satisfaction at the top, then whole-body momentum control, then end-effector tracking, then posture regularization. Lower-priority tasks are projected into the null space of higher ones. The core difficulty is that the contact Jacobians change with gait phase, requiring the solver to handle hybrid dynamics. Recent work from the RSL lab at ETH and from Carnegie Mellon’s Robotics Institute has focused on warm-starting the QP from the previous solution and using active-set methods that exploit sparsity in the constraint matrix to run the full solve in under 1ms on embedded processors.

Learning-Augmented Whole-Body Control

Pure model-based whole-body control degrades on unmodeled terrain and under significant parameter uncertainty, particularly for arm payloads that change during manipulation. Reinforcement learning policies trained in simulation have shown strong performance on locomotion but struggle with the precision required for manipulation. Hybrid approaches, where an RL policy provides reference base motions and a model-based QP handles arm tracking against those references, have produced the most reliable results in cluttered real-world trials. Unitree’s H1 platform and the Anybotics ANYmal-D with arm attachment have both been used as hardware targets in this line of work, with simulation environments in IsaacLab and MuJoCo serving as primary training grounds.

Practical Deployment Considerations

Whole-body controllers built for flat floors fail unpredictably when legs encounter steps or loose surfaces. Estimating terrain contact geometry online using foot force data and leg kinematics, then updating the contact model in the QP in real time, is necessary for deployment outside structured environments. Battery life is another underappreciated constraint: high-rate QP solves on embedded hardware draw meaningful power, and aggressive whole-body motions consume significantly more from the legs than steady locomotion. Field deployments at inspection sites and construction facilities report that manipulation budgets must be planned alongside battery budgets to avoid mid-task failures.