On General Language Understanding

David Schlangen
10/27/2023
cs.CLcs.CY

No confident code implementation found for this paper. 3 possible matches can be shown below.

Abstract

Natural Language Processing prides itself to be an empirically-minded, if not outright empiricist field, and yet lately it seems to get itself into essentialist debates on issues of meaning and measurement ("Do Large Language Models Understand Language, And If So, How Much?"). This is not by accident: Here, as everywhere, the evidence underspecifies the understanding. As a remedy, this paper sketches the outlines of a model of understanding, which can ground questions of the adequacy of current methods of measurement of model quality. The paper makes three claims: A) That different language use situation types have different characteristics, B) That language understanding is a multifaceted phenomenon, bringing together individualistic and social processes, and C) That the choice of Understanding Indicator marks the limits of benchmarking, and the beginnings of considerations of the ethics of NLP use.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Establishing Trustworthiness: Rethinking Tasks and Model Evaluation

88% match2023

The Quo Vadis of the Relationship between Language and Large Language Models

87% match2023

Language, Language Models, and What We're Talking About

86% match2026

Dialogue Games for Benchmarking Language Understanding: Motivation, Taxonomy, Strategy

84% match2023

Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?

84% match2023

Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark

84% match1 repo242023

Cite this paper

@article{schlangen2023general,
  title  = {On General Language Understanding},
  author = {David Schlangen},
  year   = {2023},
  eprint = {2310.18038},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.18038v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.18038.svg)](https://codeofpaper.com/paper/2310.18038)

Discussion