DEFT: Dexterous Fine-Tuning for Real-World Hand Policies

Aditya Kannan, Kenneth Shaw, Shikhar Bahl, Pragna Mannam, Deepak Pathak
10/30/2023
cs.ROcs.AIcs.CV

Official code: adityak77/deft-data (9 stars, last commit 2023-10-23).

Abstract

Dexterity is often seen as a cornerstone of complex manipulation. Humans are able to perform a host of skills with their hands, from making food to operating tools. In this paper, we investigate these challenges, especially in the case of soft, deformable objects as well as complex, relatively long-horizon tasks. However, learning such behaviors from scratch can be data inefficient. To circumvent this, we propose a novel approach, DEFT (DExterous Fine-Tuning for Hand Policies), that leverages human-driven priors, which are executed directly in the real world. In order to improve upon these priors, DEFT involves an efficient online optimization procedure. With the integration of human-based learning and online fine-tuning, coupled with a soft robotic hand, DEFT demonstrates success across various tasks, establishing a robust, data-efficient pathway toward general dexterous manipulation. Please see our website at https://dexterous-finetuning.github.io for video results.

View on arXivView PDF

Code Implementations(1)

adityak77/deft-dataOfficial100%

Datasets for "DEFT: Dexterous Fine-Tuning for Real-World Hand Policies" in CoRL 2023

90Oct 7, 20232 years agoNOASSERTION

Similar Papers

FAR-Dex: Few-shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulation

84% match2026

DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physics

83% match2023

From Power to Precision: Learning Fine-grained Dexterity for Multi-fingered Robotic Hands

83% match2025

World Models Can Leverage Human Videos for Dexterous Manipulation

82% match2025

MAPLE: Encoding Dexterous Robotic Manipulation Priors Learned From Egocentric Videos

82% match2025

GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Grasping

82% match2023

Cite this paper

@article{kannan2023deft,
  title  = {DEFT: Dexterous Fine-Tuning for Real-World Hand Policies},
  author = {Aditya Kannan and Kenneth Shaw and Shikhar Bahl and Pragna Mannam and Deepak Pathak},
  year   = {2023},
  eprint = {2310.19797},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.19797v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.19797.svg)](https://codeofpaper.com/paper/2310.19797)

Discussion