Defining Knowledge: Bridging Epistemology and Large Language Models

Constanza Fierro, Ruchira Dhar, Filippos Stamatiou, Nicolas Garneau, Anders Søgaard
10/3/2024
cs.CL

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

Knowledge claims are abundant in the literature on large language models (LLMs); but can we say that GPT-4 truly "knows" the Earth is round? To address this question, we review standard definitions of knowledge in epistemology and we formalize interpretations applicable to LLMs. In doing so, we identify inconsistencies and gaps in how current NLP research conceptualizes knowledge with respect to epistemological frameworks. Additionally, we conduct a survey of 100 professional philosophers and computer scientists to compare their preferences in knowledge definitions and their views on whether LLMs can really be said to know. Finally, we suggest evaluation protocols for testing knowledge in accordance to the most relevant definitions.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Epistemology of Language Models: Do Language Models Have Holistic Knowledge?

86% match2024

A Perspective on Large Language Models, Intelligent Machines, and Knowledge Acquisition

85% match2024

Measuring and Modifying Factual Knowledge in Large Language Models

83% match1 repo32023

Knowledge Mechanisms in Large Language Models: A Survey and Perspective

83% match2024

KScope: A Framework for Characterizing the Knowledge Status of Language Models

83% match2025

Decoding Knowledge in Large Language Models: A Framework for Categorization and Comprehension

83% match2025

Cite this paper

@article{fierro2024defining,
  title  = {Defining Knowledge: Bridging Epistemology and Large Language Models},
  author = {Constanza Fierro and Ruchira Dhar and Filippos Stamatiou and Nicolas Garneau and Anders Søgaard},
  year   = {2024},
  eprint = {2410.02499},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.02499v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.02499.svg)](https://codeofpaper.com/paper/2410.02499)

Discussion