LLMSteer: Improving Long-Context LLM Inference by Steering Attention on Reused Contexts

Zhuohan Gu, Jiayi Yao, Kuntai Du, Junchen Jiang
11/20/2024
cs.LGcs.CL

No confident code implementation found for this paper. 10 possible matches can be shown below.

Abstract

As large language models (LLMs) show impressive performance on complex tasks, they still struggle with longer contextual understanding and high computational costs. To balance efficiency and quality, we introduce LLMSteer, a fine-tuning-free framework that enhances LLMs through query-independent attention steering. Tested on popular LLMs and datasets, LLMSteer narrows the performance gap with baselines by 65.9% and reduces the runtime delay by up to 4.8x compared to recent attention steering methods.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 10 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Pause-Tuning for Long-Context Comprehension: A Lightweight Approach to LLM Attention Recalibration

87% match2025

LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-Tuning

87% match1 repo22025

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern

87% match2024

SEAL: Scaling to Emphasize Attention for Long-Context Retrieval

87% match2025

Squeezed Attention: Accelerating Long Context Length LLM Inference

87% match1 repo582024

A Controlled Study on Long Context Extension and Generalization in LLMs

86% match1 repo652024

Cite this paper

@article{gu2024llmsteer,
  title  = {LLMSteer: Improving Long-Context LLM Inference by Steering Attention on Reused Contexts},
  author = {Zhuohan Gu and Jiayi Yao and Kuntai Du and Junchen Jiang},
  year   = {2024},
  eprint = {2411.13009},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2411.13009v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2411.13009.svg)](https://codeofpaper.com/paper/2411.13009)

Discussion