Captioning Visualizations with Large Language Models (CVLLM): A Tutorial

Giuseppe Carenini, Jordon Johnson, Ali Salamatian
6/27/2024
cs.CLcs.AIcs.HC

No confident code implementation found for this paper.

Abstract

Automatically captioning visualizations is not new, but recent advances in large language models(LLMs) open exciting new possibilities. In this tutorial, after providing a brief review of Information Visualization (InfoVis) principles and past work in captioning, we introduce neural models and the transformer architecture used in generic LLMs. We then discuss their recent applications in InfoVis, with a focus on captioning. Additionally, we explore promising future directions in this field.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions

85% match2024

Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis

85% match2024

Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges

85% match2024

Frozen Transformers in Language Models Are Effective Visual Encoder Layers

85% match1 repo2452026

CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models

85% match2026

Attention-based transformer models for image captioning across languages: An in-depth survey and evaluation

84% match2025

Cite this paper

@article{carenini2024captioning,
  title  = {Captioning Visualizations with Large Language Models (CVLLM): A Tutorial},
  author = {Giuseppe Carenini and Jordon Johnson and Ali Salamatian},
  year   = {2024},
  eprint = {2406.19512},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.19512v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.19512.svg)](https://codeofpaper.com/paper/2406.19512)

Discussion