GPT Models Meet Robotic Applications: Co-Speech Gesturing Chat System

Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi, Jun Takamatsu, Katsushi Ikeuchi
5/10/2023
cs.ROcs.CL

Official code: microsoft/LabanotationSuite (51 stars, MIT, last commit 2024-12-20). 1 more implementation below.

Abstract

This technical paper introduces a chatting robot system that utilizes recent advancements in large-scale language models (LLMs) such as GPT-3 and ChatGPT. The system is integrated with a co-speech gesture generation system, which selects appropriate gestures based on the conceptual meaning of speech. Our motivation is to explore ways of utilizing the recent progress in LLMs for practical robotic applications, which benefits the development of both chatbots and LLMs. Specifically, it enables the development of highly responsive chatbot systems by leveraging LLMs and adds visual effects to the user interface of LLMs as an additional value. The source code for the system is available on GitHub for our in-house robot (https://github.com/microsoft/LabanotationSuite/tree/master/MSRAbotChatSimulation) and GitHub for Toyota HSR (https://github.com/microsoft/GPT-Enabled-HSR-CoSpeechGestures).

View on arXivView PDF

Code Implementations(2)

microsoft/LabanotationSuiteOfficial100%
MIT

Microsoft Applied Robotics Research Library: LabanotationSuite - open source software tools to give service robots the ability to perform human-like gestures

5120HTML, PythonJan 21, 20201 years agoMIT
microsoft/GPT-Enabled-HSR-CoSpeechGesturesOfficial100%
MIT
234PythonMar 24, 20233 years agoMIT

Similar Papers

FurChat: An Embodied Conversational Agent using LLMs, Combining Open and Closed-Domain Dialogue with Facial Expressions

88% match2023

Ain't Misbehavin' -- Using LLMs to Generate Expressive Robot Behavior in Conversations with the Tabletop Robot Haru

87% match2024

Dobby: A Conversational Service Robot Driven by GPT-4

87% match2023

GenComUI: Exploring Generative Visual Aids as Medium to Support Task-Oriented Human-Robot Communication

87% match2025

Two Heads Are Better Than One: Collaborative LLM Embodied Agents for Human-Robot Interaction

86% match2024

Exploring Large Language Models to Facilitate Variable Autonomy for Human-Robot Teaming

85% match2023

Cite this paper

@article{wake2023models,
  title  = {GPT Models Meet Robotic Applications: Co-Speech Gesturing Chat System},
  author = {Naoki Wake and Atsushi Kanehira and Kazuhiro Sasabuchi and Jun Takamatsu and Katsushi Ikeuchi},
  year   = {2023},
  eprint = {2306.01741},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.01741v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.01741.svg)](https://codeofpaper.com/paper/2306.01741)

Discussion