When Looking Changes the Grasp: Recovery Interventions in Frozen VLA Manipulation
Robot learning research · 2026
Recovery Interventions in Frozen VLA Manipulation
Motivation
A recovery supervisor must decide whether a robot has actually grasped an object before stepping in. The gripper can hide the object from the overhead camera, making that decision difficult. Moving the arm can reveal the scene, but opening the gripper while withdrawing can destroy the grasp the supervisor is trying to inspect. This project studies that physical observer effect.
Experimental approach
A task-trained, frozen SmolVLA vision-language-action policy controls an SO-101 arm in MuJoCo. Six deterministic pickup and placement scenarios branch into five conditions, giving 30 continuations: nominal execution, pause/reset only, opening in place, clearance with the gripper commanded closed, and opening plus clearance. The interrupted conditions use a common 25-step intervention protocol to separate the effects of the physical actions from the pause and reset.
Matched intervention comparison
One matched simulated continuation compares clearance with the gripper commanded closed against opening plus the same clearance motion. The aggregate results below cover all six scenarios.
Full-task completion across six reused scenarios
Nominal execution: 5 / 6 completed tasks.
Pause/reset only: 4 / 6 completed tasks.
Opening only: 5 / 6 completed tasks.
Clearance only, gripper commanded closed: 5 / 6 completed tasks.
Opening plus clearance: 0 / 6 completed tasks.
The composition of the sensing action matters. Opening plus withdrawal releases the cube and moves the arm away. A clearer view can come at the cost of the manipulation state needed to finish the task.
Equal 5/6 totals for nominal execution, opening alone, and clearance alone mask different scenario outcomes. The result does not establish that either individual action is harmless in every scene.
Passive monitoring and next steps
An exploratory offline audit of 14 nominal trajectories found that a combined wrist-camera and gripper-aperture rule flagged 2 of 6 historical failures and none of 8 historical successes. Those measurements came after closure and do not establish a timely online failure detector. Future recovery systems need to evaluate detection accuracy, false interventions, and the physical cost of intervention together.
Study scope: One simulated robot configuration, one frozen policy, and six deterministic scenarios. Privileged simulator coordinates support external controller resets, so these results do not demonstrate fully autonomous recovery or hardware reliability. One clearance-only prestate differed slightly; excluding it preserves the qualitative contrast of 4/5 versus 0/5. Physical validation remains future work.
Comments