Autonomous Manipulation Learning for Similar Deformable Objects via Only One Demonstration

Abstract

In comparison with most methods focusing on $3D$ rigid object recognition and manipulation, deformable objects are more common in our real life but attract less attention. Generally, most existing methods for deformable object manipulation suffer two issues, 1) Massive demonstration: repeating thousands of robot-object demonstrations for model training of one specific instance; 2) Poor generalization: inevitably re-training for transferring the learned skill to a similar/new instance from the same category. Therefore, we propose a category-level deformable $3D$ object manipulation framework, which could manipulate deformable $3D$ objects with only one demonstration and generalize the learned skills to new similar instances without re-training. Specifically, our proposed framework consists of two modules. The Nocs State Transform $(NST)$ module transfers the observed point clouds of the target to a pre-defined unified pose state (i.e.,Nocs state), which is the foundation for the category-level manipulation learning; the Neural Spatial Encoding $(NSE)$ module generalizes the learned skill to novel instances by encoding the category-level spatial information to pursue the expected grasping point without re-training. The relative motion path is then planned to achieve autonomous manipulation. Both the simulated results via our $\text{Cap}_{40}$ dataset and real robotic experiments justify the effectiveness of our framework.