Spectral Compressive Imaging via Unmixing-driven Subspace Diffusion Refinement

Abstract

Spectral Compressive Imaging (SCI) reconstruction is inherently ill-posed, offering multiple plausible solutions from a single observation. Traditional deterministic methods typically struggle to effectively recover high-frequency details. Although diffusion models offer promising solutions to this challenge, their application is constrained by the limited training data and high computational demands associated with multispectral images (MSIs), complicating direct training. To address these issues, we propose a novel Predict-and-unmixing-driven-Subspace-Refine framework (PSR-SCI). This framework begins with a cost-effective predictor that produces an initial, rough estimate of the MSI. Subsequently, we introduce a unmixing-driven reversible spectral embedding module that decomposes the MSI into subspace images and spectral coefficients. This decomposition facilitates the adaptation of pre-trained RGB diffusion models and focuses refinement processes on high-frequency details, thereby enabling efficient diffusion generation with minimal MSI data. Additionally, we design a high-dimensional guidance mechanism with imaging consistency to enhance the model's efficacy. The refined subspace image is then reconstructed back into an MSI using the reversible embedding, yielding the final MSI with full spectral resolution. Experimental results on the standard KAIST and zero-shot datasets NTIRE, ICVL, and Harvard show that PSR-SCI enhances visual quality and delivers PSNR and SSIM metrics comparable to existing diffusion, transformer, and deep unfolding techniques. This framework provides a robust alternative to traditional deterministic SCI reconstruction methods. [Code and models]

Key Contributions

• (i) A spectral unmixing-driven predict-and-subspace refine strategy (PSR-SCI) for SCI reconstruction that yields improved perceptual quality over deterministic methods and more efficient enhancement than typical diffusion models.
• (ii) A reversible decomposition module that utilizes hierarchical decomposition to efficiently implement spectral subspace learning while maintaining high reversibility.
• (iii) Focus on diffusion generation exclusively for high-frequency components, accelerating fine-tuning and significantly reducing required training data.
• (iv) High-dimensional guidance with SCI imaging consistency to enhance model efficacy.

Related Works

The existing framework for SCI reconstruction predominantly consists of model-based, Plug-and-Play, End-to-end (E2E), and Deep unfolding methods.

• Model-based methods depend on hand-crafted image priors but require manual parameter tuning and have limited representation capacity.
• Plug-and-play (PnP) algorithms incorporate pre-trained denoising networks into traditional model-based methods but are limited by fixed pre-trained networks.
• End-to-end (E2E) algorithms leverage CNNs to establish a mapping function but often neglect fundamental principles of SCI systems.
• Deep unfolding methods utilize multi-stage networks with interpretability through explicit characterization of image priors.
• Diffusion models - Recent works like DiffSCI utilize pre-trained denoising diffusion models for RGB images as the denoiser within the PnP framework.

Method: PSR-SCI Framework

Figure: PSR-SCI comprises (a) fast predictor with frequency splitting, (b) unmixing-based reversible embedding, and (c) subspace refinement via pretrained diffusion.

1. Snapshot Compressive Imaging and Problem Setup

In a CASSI system, an MSI \( \mathcal{X} \in \mathbb{R}^{H \times W \times B} \) is projected into a 2D measurement \( \mathcal{Y} \in \mathbb{R}^{H \times (W + d(B-1))} \) via coded spectral modulation. The imaging model can be formulated as: \[ \mathbf{y} = \mathbf{\Phi} \mathbf{x} + \mathbf{n}, \] where \( \mathbf{\Phi} \) encodes spectral-shifted masks, and \( \mathbf{x} \) is the vectorized MSI. Reconstructing \( \mathbf{x} \) from \( \mathbf{y} \) is ill-posed and benefits from strong generative priors.

2. Predict-and-Subspace-Refine Framework

We first estimate a coarse MSI \( \mathcal{X}_{\textit{init}} = \phi(\mathcal{Y}) \) via a trained predictor. Then, a learnable frequency separator \( \tau \) splits it into high- and low-frequency parts: \( (\mathcal{X}^h, \mathcal{X}^l) = \tau(\mathcal{X}_{\textit{init}}) \). The high-frequency part is embedded into subspace form: \[ (\mathcal{A}^h, E) = \psi(\mathcal{X}^h), \] where \( \mathcal{A}^h \) is a low-rank abundance map and \( E \) encodes spectral coefficients. This enables refinement via diffusion on \( \mathcal{A}^h \), with final MSI recovered as: \[ \hat{\mathcal{X}} = \psi^{-1}(\mathcal{A}^h_{\textit{diff}}, E) + \mathcal{X}^l. \]

3. Reversible Spectral Embedding (URSe)

The URSe module implements invertible unmixing via hierarchical convolutions and spectral attention, ensuring minimal information loss. Unlike direct compression, URSe guarantees structural recovery of the MSI from latent subspace.

4. Diffusion Refinement with High-Dimensional Guidance

We adapt pretrained Stable Diffusion (2.1-base) for subspace refinement. A parallel encoder allows tuning on small MSI datasets. To enforce measurement consistency, we introduce a high-dimensional guidance loss: \[ \mathcal{L} = \|\mathcal{Y} - \Phi(\psi^{-1}(\mathcal{D}(\hat{\mathcal{z}}_0), E) + \mathcal{X}^l) \|^2. \] This forms a guided reverse SDE that jointly optimizes perceptual quality and physical realism.

Experimental Results

Quantitative Evaluation

Conclusion

We introduced a new framework for spectral compressive imaging reconstruction that focuses on reconstructing high-frequency details by fine-tuning a diffusion model pre-trained on large-scale RGB images. Our empirical results demonstrate significant improvements in detail quality and superior metrics compared to current state-of-the-art methods. We believe that our work introduces a novel direction in spectral compressive imaging reconstruction and establishes a robust benchmark for future research.

Citation

@inproceedings{zeng2025spectral,
title={Spectral Compressive Imaging via Unmixing-driven Subspace Diffusion Refinement},
author={Zeng, Haijin and Sun, Benteng and Chen, Yongyong and Su, Jingyong and Xu, Yong},
booktitle={The Thirteenth International Conference on Learning Representations}
    year={2025}
}