TEACh: Task-driven Embodied Agents that Chat

Aishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava, Patrick Lange, Anjali Narayan-Chen +4 more
10/1/2021
cs.CVcs.AIcs.CL

Official code: alexa/teach (146 stars, last commit 2024-05-06).

Abstract

Robots operating in human spaces must be able to engage in natural language interaction with people, both understanding and executing instructions, and using conversation to resolve ambiguity and recover from mistakes. To study this, we introduce TEACh, a dataset of over 3,000 human--human, interactive dialogues to complete household tasks in simulation. A Commander with access to oracle information about a task communicates in natural language with a Follower. The Follower navigates through and interacts with the environment to complete tasks varying in complexity from "Make Coffee" to "Prepare Breakfast", asking questions and getting additional information from the Commander. We propose three benchmarks using TEACh to study embodied intelligence challenges, and we evaluate initial models' abilities in dialogue understanding, language grounding, and task execution.

View on arXivView PDF

Code Implementations(1)

alexa/teachOfficial100%

TEACh is a dataset of human-human interactive dialogues to complete tasks in a simulated household environment.

14627Python, Jupyter NotebookSep 24, 20212 years ago

Similar Papers

MEIA: Multimodal Embodied Perception and Interaction in Unknown Environments

84% match1 repo1.1k2024

User Simulation with Large Language Models for Evaluating Task-Oriented Dialogue

84% match2023

Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions

84% match1 repo32023

SpokenUS: A Spoken User Simulator for Task-Oriented Dialogue

84% match2026

Inner Monologue: Embodied Reasoning through Planning with Language Models

84% match2022

Agentic Persona Control and Task State Tracking for Realistic User Simulation in Interactive Scenarios

84% match2025

Cite this paper

@article{padmakumar2021teach,
  title  = {TEACh: Task-driven Embodied Agents that Chat},
  author = {Aishwarya Padmakumar and Jesse Thomason and Ayush Shrivastava and Patrick Lange and Anjali Narayan-Chen and Spandana Gella and Robinson Piramuthu and Gokhan Tur and Dilek Hakkani-Tur},
  year   = {2021},
  eprint = {2110.00534},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2110.00534}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2110.00534.svg)](https://codeofpaper.com/paper/2110.00534)

Discussion