Is ChatGPT A Good Keyphrase Generator? A Preliminary Study

Mingyang Song, Haiyun Jiang, Shuming Shi, Songfang Yao, Shilong Lu +3 more
3/23/2023
cs.CL

No official code found. Best implementation: MySong7NLPer/ChatGPT_as_Keyphrase_Generator (6 stars, last commit 2024-01-08).

Abstract

The emergence of ChatGPT has recently garnered significant attention from the computational linguistics community. To demonstrate its capabilities as a keyphrase generator, we conduct a preliminary evaluation of ChatGPT for the keyphrase generation task. We evaluate its performance in various aspects, including keyphrase generation prompts, keyphrase generation diversity, and long document understanding. Our evaluation is based on six benchmark datasets, and we adopt the prompt suggested by OpenAI while extending it to six candidate prompts. We find that ChatGPT performs exceptionally well on all six candidate prompts, with minor performance differences observed across the datasets. Based on our findings, we conclude that ChatGPT has great potential for keyphrase generation. Moreover, we discover that ChatGPT still faces challenges when it comes to generating absent keyphrases. Meanwhile, in the final section, we also present some limitations and future expansions of this report.

View on arXivView PDF

Code Implementations(1)

MySong7NLPer/ChatGPT_as_Keyphrase_GeneratorCommunity100%

Data for ArXiv 2023 paper "Is ChatGPT A Good Keyphrase Generator? A Preliminary Study".

60Mar 9, 20232 years ago
keyphrase-generationlarge-language-model

Similar Papers

ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task

88% match2023

KPEval: Towards Fine-Grained Semantic-Based Keyphrase Evaluation

88% match1 repo132023

An Analysis of Datasets, Metrics and Models in Keyphrase Generation

87% match1 repo2025

Enhancing Keyphrase Generation by BART Finetuning with Splitting and Shuffling

87% match2023

Cross-Domain Robustness of Transformer-based Keyphrase Generation

86% match2023

BibRank: Automatic Keyphrase Extraction Platform Using~Metadata

86% match2023

Cite this paper

@article{song2023chatgpt,
  title  = {Is ChatGPT A Good Keyphrase Generator? A Preliminary Study},
  author = {Mingyang Song and Haiyun Jiang and Shuming Shi and Songfang Yao and Shilong Lu and Yi Feng and Huafeng Liu and Liping Jing},
  year   = {2023},
  eprint = {2303.13001},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.13001v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.13001.svg)](https://codeofpaper.com/paper/2303.13001)

Discussion