Speech Is Not Enough: Interpreting Nonverbal Indicators of Common Knowledge and Engagement

Derek Palmer, Yifan Zhu, Kenneth Lai, Hannah VanderHoeven, Mariah Bradford +6 more
12/8/2024
cs.CLcs.AI

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Our goal is to develop an AI Partner that can provide support for group problem solving and social dynamics. In multi-party working group environments, multimodal analytics is crucial for identifying non-verbal interactions of group members. In conjunction with their verbal participation, this creates an holistic understanding of collaboration and engagement that provides necessary context for the AI Partner. In this demo, we illustrate our present capabilities at detecting and tracking nonverbal behavior in student task-oriented interactions in the classroom, and the implications for tracking common ground and engagement.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

The Impact of Background Speech on Interruption Detection in Collaborative Groups

81% match2025

MimeQA: Towards Socially-Intelligent Nonverbal Foundation Models

81% match1 repo52025

TRACE: Real-Time Multimodal Common Ground Tracking in Situated Collaborative Dialogues

80% match2025

Dude, where's my utterance? Evaluating the effects of automatic segmentation and transcription on CPS detection

80% match2025

Speaking Beyond Language: A Large-Scale Multimodal Dataset for Learning Nonverbal Cues from Video-Grounded Dialogues

79% match1 repo62025

Common Ground Tracking in Multimodal Dialogue

79% match1 repo42024

Cite this paper

@article{palmer2024speech,
  title  = {Speech Is Not Enough: Interpreting Nonverbal Indicators of Common Knowledge and Engagement},
  author = {Derek Palmer and Yifan Zhu and Kenneth Lai and Hannah VanderHoeven and Mariah Bradford and Ibrahim Khebour and Carlos Mabrey and Jack Fitzgerald and Nikhil Krishnaswamy and Martha Palmer and James Pustejovsky},
  year   = {2024},
  eprint = {2412.05797},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.05797v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.05797.svg)](https://codeofpaper.com/paper/2412.05797)

Discussion