TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning

Marius Dragoi, Ioana Pintilie, Alexandra Dragomir, Antonio Barbalau, Florin Brad
6/4/2026
cs.LG

No confident code implementation found for this paper.

Abstract

Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning. In this paper we introduce TailLoR, which utilizes the singular bases U and V of the pre-trained weights as a fixed reference frame to learn a low-rank update applied to the singular value matrix. A soft spectral penalty discourages updates aligned with dominant singular directions, reducing interference while routing fine-grained adaptation into the highly flexible, long-tail spectral coordinates.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Spectral Adapter: Fine-Tuning in Spectral Space

85% match1 repo122024

SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning

84% match2026

ALLoRA: Adaptive Learning Rate Mitigates LoRA Fatal Flaws

82% match2024

SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors

82% match1 repo352026

PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer

82% match2026

The Primacy of Magnitude in Low-Rank Adaptation

82% match2025

Cite this paper

@article{dragoi2026taillor,
  title  = {TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning},
  author = {Marius Dragoi and Ioana Pintilie and Alexandra Dragomir and Antonio Barbalau and Florin Brad},
  year   = {2026},
  eprint = {2606.06494},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2606.06494v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2606.06494.svg)](https://codeofpaper.com/paper/2606.06494)

Discussion