Large Language Models as Zero-Shot Conversational Recommenders

Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang +4 more
8/19/2023
cs.IRcs.AI

No official code found. Best implementation: AaronHeee/LLMs-as-Zero-Shot-Conversational-RecSys (86 stars, last commit 2023-08-22).

Abstract

In this paper, we present empirical studies on conversational recommendation tasks using representative large language models in a zero-shot setting with three primary contributions. (1) Data: To gain insights into model behavior in "in-the-wild" conversational recommendation scenarios, we construct a new dataset of recommendation-related conversations by scraping a popular discussion website. This is the largest public real-world conversational recommendation dataset to date. (2) Evaluation: On the new dataset and two existing conversational recommendation datasets, we observe that even without fine-tuning, large language models can outperform existing fine-tuned conversational recommendation models. (3) Analysis: We propose various probing tasks to investigate the mechanisms behind the remarkable performance of large language models in conversational recommendation. We analyze both the large language models' behaviors and the characteristics of the datasets, providing a holistic understanding of the models' effectiveness, limitations and suggesting directions for the design of future conversational recommenders

View on arXivView PDF

Code Implementations(1)

AaronHeee/LLMs-as-Zero-Shot-Conversational-RecSysCommunity100%

Evaluation data, LLMs query code and results for "Large Language Models as Zero-Shot Conversational Recommenders" on CIKM 2023.

8610PythonAug 17, 20233 years ago

Similar Papers

Large Language Models as Conversational Movie Recommenders: A User Study

88% match2024

Improving Conversational Recommendation Systems via Bias Analysis and Language-Model-Enhanced Data Augmentation

88% match1 repo82023

Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models

87% match2 repos642023

Large Language Model Augmented Narrative Driven Recommendations

87% match1 repo82023

USB-Rec: An Effective Framework for Improving Conversational Recommendation Capability of Large Language Model

87% match1 repo242025

ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation

87% match2 repos582023

Cite this paper

@article{he2023large,
  title  = {Large Language Models as Zero-Shot Conversational Recommenders},
  author = {Zhankui He and Zhouhang Xie and Rahul Jha and Harald Steck and Dawen Liang and Yesu Feng and Bodhisattwa Prasad Majumder and Nathan Kallus and Julian McAuley},
  year   = {2023},
  eprint = {2308.10053},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2308.10053v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2308.10053.svg)](https://codeofpaper.com/paper/2308.10053)

Discussion