Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions

Shrestha Mohanty, Negar Arabzadeh, Julia Kiseleva, Artem Zholus, Milagro Teruel +4 more
5/18/2023
cs.AI

No official code found. Best implementation: iglu-contest/nlp-baselines-2022 (3 stars, MIT, last commit 2023-05-31).

Abstract

Human intelligence's adaptability is remarkable, allowing us to adjust to new tasks and multi-modal environments swiftly. This skill is evident from a young age as we acquire new abilities and solve problems by imitating others or following natural language instructions. The research community is actively pursuing the development of interactive "embodied agents" that can engage in natural conversations with humans and assist them with real-world tasks. These agents must possess the ability to promptly request feedback in case communication breaks down or instructions are unclear. Additionally, they must demonstrate proficiency in learning new vocabulary specific to a given domain. In this paper, we made the following contributions: (1) a crowd-sourcing tool for collecting grounded language instructions; (2) the largest dataset of grounded language instructions; and (3) several state-of-the-art baselines. These contributions are suitable as a foundation for further research.

View on arXivView PDF

Code Implementations(1)

iglu-contest/nlp-baselines-2022Community100%
MIT

The repository contains the tope performing NLP solutions for IGLU 2022 competition.

30Python, Jupyter Notebook3 years agoMIT

Similar Papers

BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning

85% match1 repo7712018

TEACh: Task-driven Embodied Agents that Chat

84% match1 repo1462021

GUI Agents with Foundation Models: A Comprehensive Survey

84% match2024

COLLEAGUE.SKILL: Automated AI Skill Generation via Expert Knowledge Distillation

84% match2026

Improving Knowledge Extraction from LLMs for Task Learning through Agent Analysis

84% match1 repo42023

See and Think: Embodied Agent in Virtual Environment

84% match1 repo422023

Cite this paper

@article{mohanty2023transforming,
  title  = {Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions},
  author = {Shrestha Mohanty and Negar Arabzadeh and Julia Kiseleva and Artem Zholus and Milagro Teruel and Ahmed Awadallah and Yuxuan Sun and Kavya Srinet and Arthur Szlam},
  year   = {2023},
  eprint = {2305.10783},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.10783v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.10783.svg)](https://codeofpaper.com/paper/2305.10783)

Discussion