Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models

Sitao Cheng, Liangming Pan, Xunjian Yin, Xinyi Wang, William Yang Wang
10/10/2024
cs.CL

Official code: sitaocheng/Knowledge_Interplay (12 stars, last commit 2025-07-17).

Abstract

Large language models (LLMs) encode vast amounts of knowledge during pre-training (parametric knowledge, or PK) and can further be enhanced by incorporating contextual knowledge (CK). Can LLMs effectively integrate their internal PK with external CK to solve complex problems? In this paper, we investigate the dynamic interaction between PK and CK, categorizing their relationships into four types: Supportive, Complementary, Conflicting, and Irrelevant. To support this investigation, we introduce ECHOQA, a benchmark spanning scientific, factual, and commonsense knowledge. Our results show that LLMs tend to suppress their PK when contextual information is available, even when it is complementary or irrelevant. While tailored instructions can encourage LLMs to rely more on their PK, they still struggle to fully leverage it. These findings reveal a key vulnerability in LLMs, raising concerns about their reliability in knowledge-intensive tasks. Resources are available at https://github.com/sitaocheng/Knowledge_Interplay

View on arXivView PDF

Code Implementations(1)

sitaocheng/Knowledge_InterplayOfficial100%

Code and dataset for EchoQA, testing LLMs ability to leverage parametric knowledge when contextual knowledge is available.

120PythonOct 9, 20241 years ago

Similar Papers

Probing Language Models on Their Knowledge Source

91% match1 repo42024

Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge: A keynote at ECIR 2025

90% match2026

When Context Leads but Parametric Memory Follows in Large Language Models

88% match1 repo12024

Evaluating the External and Parametric Knowledge Fusion of Large Language Models

88% match2024

Continuously Steering LLMs Sensitivity to Contextual Knowledge with Proxy Models

88% match1 repo32025

"Lost-in-the-Later": Framework for Quantifying Contextual Grounding in Large Language Models

87% match2025

Cite this paper

@article{cheng2024understanding,
  title  = {Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models},
  author = {Sitao Cheng and Liangming Pan and Xunjian Yin and Xinyi Wang and William Yang Wang},
  year   = {2024},
  eprint = {2410.08414},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.08414v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.08414.svg)](https://codeofpaper.com/paper/2410.08414)

Discussion