Anonymous Submission

SRPM: Spatial Rehearsal and Persistent Memory via Latent Space for Whole-Body Mobile Manipulation

Anonymous Authors

75.8% MS-HAB mean success
+18.8% Simulation gain over SOTA
+13.8% Real-world gain over baseline
Real-world whole-body mobile manipulation tasks and success-rate comparison.
SRPM is evaluated on whole-body mobile manipulation tasks that require the base and arm to act together, including open drawer, clean table, mobile pick, and box pick-and-place.

Abstract

Compared to tabletop tasks with fixed bases and cameras, mobile manipulation faces significant whole-body coordination challenges. Any mismatch between mobile base and arm movements is amplified through the robot's kinematic chain, leading to significant execution errors. Moreover, frequent viewpoint changes caused by base motion disrupt the policy's perceptual consistency, making it difficult for the robot to stay focused on the target. We propose SRPM, a latent-space Spatial Rehearsal and Persistent Memory framework for whole-body mobile manipulation. SRPM introduces Cascaded Predictive Coordination to explicitly model hierarchical base-arm dependencies, Spatial Rehearsal to anticipate base-motion-induced changes in latent 3D geometry, and Latent Intention Memory to maintain target consistency under occlusions and rapid viewpoint shifts. Extensive simulation and real-world experiments show that SRPM achieves 18.8% and 13.8% gains over state-of-the-art methods, respectively.

Method

SRPM combines cascaded base-to-arm prediction, spatial rehearsal, and persistent latent memory for coordinated mobile manipulation.

Overview of the SRPM pipeline.
Overview of SRPM. The mobility branch predicts base intentions from 2D observations and local maps. These intentions condition a manipulation branch through Latent Intention Memory and Spatial Rehearsal, enabling arm prediction in a base-motion-aware latent reference frame.

Cascaded Predictive Coordination

SRPM first predicts coarse-grained base actions and then conditions fine-grained arm actions on the latent mobility intention, making the manipulation branch aware of planned base motion.

Spatial Rehearsal

Instead of hallucinating future states, SRPM applies deterministic geometric transformations induced by base motion and aligns rehearsed point-cloud features with transformed 3D references.

Persistent Memory

Latent Intention Memory retrieves and consolidates historical mobility intentions, allowing the policy to maintain target consistency during occlusion, target loss, and large viewpoint changes.

Spatial rehearsal process.
Spatial Rehearsal models base-induced point-cloud transformations before planning fine-grained arm motion.
Latent Intention Memory mechanism.
Latent Intention Memory retrieves historical context when the target becomes unreliable or temporarily invisible.

Results

SRPM improves mobile manipulation success in simulation and real-world experiments.

MS-HAB benchmark task visualization.
MS-HAB benchmark tasks include pick/place, opening, and closing behaviors across room-scale workspaces. Red boxes indicate target objects and cyan trajectories show mobile-base movement.
MS-HAB Simulation Results, Success Rate (%)
Method Pick Apple Place Apple Open Door Pick Bowl Place Bowl Open Drawer Close Drawer Mean
DP9.057.767.220.167.727.591.048.6
ACT*35.444.467.266.737.634.488.453.4
AC-DiT33.333.390.736.017.081.397.355.6
DSPv2*15.370.477.855.085.740.798.463.3
SRPM36.580.183.665.683.184.796.875.8

* denotes modality-aligned baseline variants. The table shows representative rows from the paper for readability.

Visualization of Latent Intention Memory under target loss.
LIM helps the policy recover target focus after wrist-view target loss and drastic camera movement.

Ablation Study

VariantMean
Basic58.6
Basic + 3D61.6
Basic + 3D + CAP63.7
Basic + 3D + CPC68.4
Basic + 3D + CPC + LIM70.6

Videos

Supplementary real-world demonstrations for the four whole-body mobile manipulation tasks.

Open Drawer

Clean Table

Mobile Pick

Box Pick & Place