Diffusion Timbre Transfer Via Mutual Information Guided Inpainting

Ching Ho Lee, Javier Nistal, Stefan Lattner, Marco Pasini, George Fazekas
1/3/2026
cs.SDcs.AIeess.AS

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

We study timbre transfer as an inference-time editing problem for music audio. Starting from a strong pre-trained latent diffusion model, we introduce a lightweight procedure that requires no additional training: (i) a dimension-wise noise injection that targets latent channels most informative of instrument identity, and (ii) an early-step clamping mechanism that re-imposes the input's melodic and rhythmic structure during reverse diffusion. The method operates directly on audio latents and is compatible with text/audio conditioning (e.g., CLAP). We discuss design choices,analyze trade-offs between timbral change and structural preservation, and show that simple inference-time controls can meaningfully steer pre-trained models for style-transfer use cases.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Diff-TONE: Timestep Optimization for iNstrument Editing in Text-to-Music Diffusion Models

88% match2025

Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfer

87% match2024

Combining audio control and style transfer using latent diffusion

87% match1 repo672024

DITTO: Diffusion Inference-Time T-Optimization for Music Generation

86% match2024

MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models

84% match1 repo492024

Controllable Music Production with Diffusion Models and Guidance Gradients

84% match2023

Cite this paper

@article{lee2026diffusion,
  title  = {Diffusion Timbre Transfer Via Mutual Information Guided Inpainting},
  author = {Ching Ho Lee and Javier Nistal and Stefan Lattner and Marco Pasini and George Fazekas},
  year   = {2026},
  eprint = {2601.01294},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2601.01294v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2601.01294.svg)](https://codeofpaper.com/paper/2601.01294)

Discussion