HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models

Abstract

Hyperspectral image (HSI) restoration aims at recov-ering clean images from degraded observations and plays a vital role in downstream tasks. Existing model-based methods have limitations in accurately modeling the com-plex image characteristics with handcraft priors, and deep learning-based methods suffer from poor generalization ability. To alleviate these issues, this paper proposes an unsupervised HSI restoration framework with pre-trained diffusion model (HIR-Diff), which restores the clean HSls from the product of two low-rank components, i.e., the re-duced image and the coefficient matrix. Specifically, the re-duced image, which has a low spectral dimension, lies in the image field and can be inferred from our improved diffusion model where a new guidance function with total variation (TV) prior is designed to ensure that the reduced image can be well sampled. The coefficient matrix can be effectively pre-estimated based on singular value decomposition (SVD) and rank-revealing QR (RRQR) factorization. Fur-thermore, a novel exponential noise schedule is proposed to accelerate the restoration process (about 5 x acceleration for denoising) with little performance decrease. Ex-tensive experimental results validate the superiority of our method in both performance and speed on a variety of HSI restoration tasks, including HSI denoising, noisy HSI super-resolution, and noisy HSI inpainting. The code is available at https://github.com/LiPang/HIRDiff.