Can Language Model Understand Word Semantics as A Chatbot? An Empirical Study of Language Model Internal External Mismatch

Jinman Zhao, Xueyan Zhang, Xingyu Yue, Weizhe Chen, Zifan Qian +1 more
9/21/2024
cs.CL

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Current common interactions with language models is through full inference. This approach may not necessarily align with the model's internal knowledge. Studies show discrepancies between prompts and internal representations. Most focus on sentence understanding. We study the discrepancy of word semantics understanding in internal and external mismatch across Encoder-only, Decoder-only, and Encoder-Decoder pre-trained language models.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

On the Semantics of Large Language Models

83% match2025

Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models

82% match2024

Understanding the Inner Workings of Language Models Through Representation Dissimilarity

81% match2023

Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Models

81% match2026

Holmes: A Benchmark to Assess the Linguistic Competence of Language Models

81% match2024

Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts?

81% match2024

Cite this paper

@article{zhao2024language,
  title  = {Can Language Model Understand Word Semantics as A Chatbot? An Empirical Study of Language Model Internal External Mismatch},
  author = {Jinman Zhao and Xueyan Zhang and Xingyu Yue and Weizhe Chen and Zifan Qian and Ruiyu Wang},
  year   = {2024},
  eprint = {2409.13972},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.13972v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.13972.svg)](https://codeofpaper.com/paper/2409.13972)

Discussion