RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models

Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou +12 more
10/1/2023
cs.CLcs.AI

No official code found. Best implementation: InteractiveNLP-Team/RoleLLM-public (527 stars, last commit 2024-10-11).

Abstract

The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters. However, the closed-source nature of state-of-the-art LLMs and their general-purpose training limit role-playing optimization. In this paper, we introduce RoleLLM, a framework to benchmark, elicit, and enhance role-playing abilities in LLMs. RoleLLM comprises four stages: (1) Role Profile Construction for 100 roles; (2) Context-Based Instruction Generation (Context-Instruct) for role-specific knowledge extraction; (3) Role Prompting using GPT (RoleGPT) for speaking style imitation; and (4) Role-Conditioned Instruction Tuning (RoCIT) for fine-tuning open-source models along with role customization. By Context-Instruct and RoleGPT, we create RoleBench, the first systematic and fine-grained character-level benchmark dataset for role-playing with 168,093 samples. Moreover, RoCIT on RoleBench yields RoleLLaMA (English) and RoleGLM (Chinese), significantly enhancing role-playing abilities and even achieving comparable results with RoleGPT (using GPT-4).

View on arXivView PDF

Code Implementations(1)

InteractiveNLP-Team/RoleLLM-publicCommunity100%

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models

52718Oct 1, 20232 years ago

Similar Papers

RMTBench: Benchmarking LLMs Through Multi-Turn User-Centric Role-Playing

88% match2025

Better Zero-Shot Reasoning with Role-Play Prompting

87% match1 repo362023

AudioRole: An Audio Dataset for Character Role-Playing in Large Language Models

87% match1 repo12025

OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas

87% match2025

CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation

87% match1 repo3012024

Test-Time-Matching: Decouple Personality, Memory, and Linguistic Style in LLM-based Role-Playing Language Agent

86% match1 repo152025

Cite this paper

@article{wang2023rolellm,
  title  = {RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models},
  author = {Zekun Moore Wang and Zhongyuan Peng and Haoran Que and Jiaheng Liu and Wangchunshu Zhou and Yuhan Wu and Hongcheng Guo and Ruitong Gan and Zehao Ni and Jian Yang and Man Zhang and Zhaoxiang Zhang and Wanli Ouyang and Ke Xu and Stephen W. Huang and Jie Fu and Junran Peng},
  year   = {2023},
  eprint = {2310.00746},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.00746v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.00746.svg)](https://codeofpaper.com/paper/2310.00746)

Discussion