Are Large Language Models Actually Good at Text Style Transfer?

Sourabrata Mukherjee, Atul Kr. Ojha, Ondřej Dušek
6/9/2024
cs.CL

No official code found. Best implementation: souro/tst_llm (3 stars, MIT, last commit 2024-09-27).

Abstract

We analyze the performance of large language models (LLMs) on Text Style Transfer (TST), specifically focusing on sentiment transfer and text detoxification across three languages: English, Hindi, and Bengali. Text Style Transfer involves modifying the linguistic style of a text while preserving its core content. We evaluate the capabilities of pre-trained LLMs using zero-shot and few-shot prompting as well as parameter-efficient finetuning on publicly available datasets. Our evaluation using automatic metrics, GPT-4 and human evaluations reveals that while some prompted LLMs perform well in English, their performance in on other languages (Hindi, Bengali) remains average. However, finetuning significantly improves results compared to zero-shot and few-shot prompting, making them comparable to previous state-of-the-art. This underscores the necessity of dedicated datasets and specialized models for effective TST.

View on arXivView PDF

Code Implementations(1)

souro/tst_llmCommunity100%
MIT

Exploration of Text Style Transfer Tasks Using Large Language Models in multilingual and multi-styles settings

30Shell, PythonJun 8, 20242 years agoMIT

Similar Papers

Multilingual Text Style Transfer: Datasets & Models for Indian Languages

92% match2 repos2024

Evaluating Text Style Transfer Evaluation: Are There Any Reliable Metrics?

90% match2025

Evaluating Text Style Transfer: A Nine-Language Benchmark for Text Detoxification

90% match2025

Text Style Transfer with Parameter-efficient LLM Finetuning and Round-trip Translation

87% match2026

Unsupervised Text Style Transfer via LLMs and Attention Masking with Multi-way Interactions

83% match2024

Steering Large Language Models with Register Analysis for Arbitrary Style Transfer

83% match2025

Cite this paper

@article{mukherjee2024large,
  title  = {Are Large Language Models Actually Good at Text Style Transfer?},
  author = {Sourabrata Mukherjee and Atul Kr. Ojha and Ondřej Dušek},
  year   = {2024},
  eprint = {2406.05885},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.05885v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.05885.svg)](https://codeofpaper.com/paper/2406.05885)

Discussion