ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuation

Yanzhou Pan, Huawei Lin, Yide Ran, Jiamin Chen, Xiaodong Yu +3 more
3/2/2025
cs.LG

No official code found. Best implementation: huawei-lin/RapidIn (21 stars, last commit 2026-03-10).

Abstract

Large Language Models (LLMs) heavily rely on high-quality training data, making data valuation crucial for optimizing model performance, especially when working within a limited budget. In this work, we aim to offer a third-party data valuation approach that benefits both data providers and model developers. We introduce a linearized future influence kernel (LinFiK), which assesses the value of individual data samples in improving LLM performance during training. We further propose ALinFiK, a learning strategy to approximate LinFiK, enabling scalable data valuation. Our comprehensive evaluations demonstrate that this approach surpasses existing baselines in effectiveness and efficiency, demonstrating significant scalability advantages as LLM parameters increase.

View on arXivView PDF

Code Implementations(1)

huawei-lin/RapidInCommunity100%

RapidIn: Scalable Influence Estimation for Large Language Models (LLMs). The implementation for paper "Token-wise Influential Training Data Retrieval for Large Language Models" (Accepted on ACL 2024).

215PythonMay 16, 20247 months agoNOASSERTION

Similar Papers

What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions

87% match1 repo1472024

Efficient Forward-Only Data Valuation for Pretrained LLMs and VLMs

85% match2025

Data Valuation for LLM Fine-Tuning: Efficient Shapley Value Approximation via Language Model Arithmetic

84% match2025

Data Value in the Age of Scaling: Understanding LLM Scaling Dynamics Under Real-Synthetic Data Mixtures

83% match2025

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs

83% match2024

Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference

82% match2024

Cite this paper

@article{pan2025alinfik,
  title  = {ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuation},
  author = {Yanzhou Pan and Huawei Lin and Yide Ran and Jiamin Chen and Xiaodong Yu and Weijie Zhao and Denghui Zhang and Zhaozhuo Xu},
  year   = {2025},
  eprint = {2503.01052},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2503.01052v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2503.01052.svg)](https://codeofpaper.com/paper/2503.01052)

Discussion