SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM's SVG Editing Capabilities

Kunato Nishina, Yusuke Matsui
4/21/2024
cs.CV

Official code: mti-lab/SVGEditBench (38 stars, MIT, last commit 2024-10-17).

Abstract

Text-to-image models have shown progress in recent years. Along with this progress, generating vector graphics from text has also advanced. SVG is a popular format for vector graphics, and SVG represents a scene with XML text. Therefore, Large Language Models can directly process SVG code. Taking this into account, we focused on editing SVG with LLMs. For quantitative evaluation of LLMs' ability to edit SVG, we propose SVGEditBench. SVGEditBench is a benchmark for assessing the LLMs' ability to edit SVG code. We also show the GPT-4 and GPT-3.5 results when evaluated on the proposed benchmark. In the experiments, GPT-4 showed superior performance to GPT-3.5 both quantitatively and qualitatively. The dataset is available at https://github.com/mti-lab/SVGEditBench.

View on arXivView PDF

Code Implementations(1)

mti-lab/SVGEditBenchOfficial100%
MIT

A benchmark dataset for evaluating LLM's SVG editing capabilities

385PythonApr 10, 20242 years agoMIT

Similar Papers

Leveraging Large Language Models For Scalable Vector Graphics Processing: A Review

90% match2025

VectorEdits: A Dataset and Benchmark for Instruction-Based Editing of Vector Graphics

85% match2025

DualEdit: Dual Editing for Knowledge Updating in Vision-Language Models

84% match1 repo102025

InternSVG: Towards Unified SVG Tasks with Multimodal Large Language Models

84% match1 repo1172025

DreamOmni: Unified Image Generation and Editing

84% match2026

SVGDreamer++: Advancing Editability and Diversity in Text-Guided SVG Generation

83% match1 repo4522024

Cite this paper

@article{nishina2024svgeditbench,
  title  = {SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM's SVG Editing Capabilities},
  author = {Kunato Nishina and Yusuke Matsui},
  year   = {2024},
  eprint = {2404.13710},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.13710v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.13710.svg)](https://codeofpaper.com/paper/2404.13710)

Discussion