Gaussian process implicit surfaces for shape estimation and grasping

TLDR

An adequate object shape representation must support probabilistic sensory fusion and enable efficient grasp and motion generation. The study proposes using Gaussian process implicit surface potentials to represent object shapes. Gaussian processes are conditioned on multimodal sensory data to generate an implicit surface that serves as a grasp controller attractor, validated in simulation and on a 7‑DoF robot arm and hand. The controller yields smooth, phase‑free reach and grasp trajectories.

Abstract

The choice of an adequate object shape representation is critical for efficient grasping and robot manipulation. A good representation has to account for two requirements: it should allow uncertain sensory fusion in a probabilistic way and it should serve as a basis for efficient grasp and motion generation. We consider Gaussian process implicit surface potentials as object shape representations. Sensory observations condition the Gaussian process such that its posterior mean defines an implicit surface which becomes an estimate of the object shape. Uncertain visual, haptic and laser data can equally be fused in the same Gaussian process shape estimate. The resulting implicit surface potential can then be used directly as a basis for a reach and grasp controller, serving as an attractor for the grasp end-effectors and steering the orientation of contact points. Our proposed controller results in a smooth reach and grasp trajectory without strict separation of phases. We validate the shape estimation using Gaussian processes in a simulation on randomly sampled shapes and the grasp controller on a real robot with 7DoF arm and 7DoF hand.