Evaluating the Knowledge Base Completion Potential of GPT

Blerta Veseli, Simon Razniewski, Jan-Christoph Kalo, Gerhard Weikum
10/23/2023
cs.CLcs.AI

No confident code implementation found for this paper. 5 possible matches can be shown below.

Abstract

Structured knowledge bases (KBs) are an asset for search engines and other applications, but are inevitably incomplete. Language models (LMs) have been proposed for unsupervised knowledge base completion (KBC), yet, their ability to do this at scale and with high accuracy remains an open question. Prior experimental studies mostly fall short because they only evaluate on popular subjects, or sample already existing facts from KBs. In this work, we perform a careful evaluation of GPT's potential to complete the largest public KB: Wikidata. We find that, despite their size and capabilities, models like GPT-3, ChatGPT and GPT-4 do not achieve fully convincing results on this task. Nonetheless, they provide solid improvements over earlier approaches with smaller LMs. In particular, we show that, with proper thresholding, GPT-3 enables to extend Wikidata by 27M facts at 90% precision.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 5 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Evaluating Language Models for Knowledge Base Completion

91% match1 repo72023

Knowledge Base Completion for Long-Tail Entities

84% match2023

Expanding the Vocabulary of BERT for Knowledge Base Construction

82% match2023

KGQuiz: Evaluating the Generalization of Encoded Knowledge in Large Language Models

80% match1 repo82023

Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models

79% match1 repo442023

Completeness, Recall, and Negation in Open-World Knowledge Bases: A Survey

79% match2023

Cite this paper

@article{veseli2023evaluating,
  title  = {Evaluating the Knowledge Base Completion Potential of GPT},
  author = {Blerta Veseli and Simon Razniewski and Jan-Christoph Kalo and Gerhard Weikum},
  year   = {2023},
  eprint = {2310.14771},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.14771v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.14771.svg)](https://codeofpaper.com/paper/2310.14771)

Discussion