Open-TeleVision: Teleoperation with Immersive Active Visual Feedback

Xuxin Cheng, Jialong Li, Shiqi Yang, Ge Yang, Xiaolong Wang
7/1/2024
cs.ROcs.HCcs.LG

No official code found. Best implementation: OpenTeleVision/TeleVision (1,290 stars, last commit 2024-09-27).

Abstract

Teleoperation serves as a powerful method for collecting on-robot data essential for robot learning from demonstrations. The intuitiveness and ease of use of the teleoperation system are crucial for ensuring high-quality, diverse, and scalable data. To achieve this, we propose an immersive teleoperation system Open-TeleVision that allows operators to actively perceive the robot's surroundings in a stereoscopic manner. Additionally, the system mirrors the operator's arm and hand movements on the robot, creating an immersive experience as if the operator's mind is transmitted to a robot embodiment. We validate the effectiveness of our system by collecting data and training imitation learning policies on four long-horizon, precise tasks (Can Sorting, Can Insertion, Folding, and Unloading) for 2 different humanoid robots and deploy them in the real world. The system is open-sourced at: https://robot-tv.github.io/

View on arXivView PDF

Code Implementations(1)

OpenTeleVision/TeleVisionCommunity71%

[CoRL 2024] Open-TeleVision: Teleoperation with Immersive Active Visual Feedback

1,290136HTML, CMakeApr 25, 20242 years agoNOASSERTION

Similar Papers

OpenVR: Teleoperation for Manipulation

86% match2023

Tool-as-Interface: Learning Robot Policies from Observing Human Tool Use

84% match2025

Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models

84% match2025

XRoboToolkit: A Cross-Platform Framework for Robot Teleoperation

84% match2025

A Vision-Based Shared-Control Teleoperation Scheme for Controlling the Robotic Arm of a Four-Legged Robot

83% match2025

Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning

83% match2024

Cite this paper

@article{cheng2024opentelevision,
  title  = {Open-TeleVision: Teleoperation with Immersive Active Visual Feedback},
  author = {Xuxin Cheng and Jialong Li and Shiqi Yang and Ge Yang and Xiaolong Wang},
  year   = {2024},
  eprint = {2407.01512},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.01512v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.01512.svg)](https://codeofpaper.com/paper/2407.01512)

Discussion