DSwinIR: Rethinking Window-based Attention for Image Restoration

Image restoration has witnessed significant advancements with the development of deep learning models. Transformer-based models, particularly those using window-based self-attention, have become a dominant force. However, their performance is constrained by the rigid, non-overlapping window partitioning scheme, which leads to $\mathit{\text{insufficient}}\mathit{\text{feature}}\mathit{\text{interaction}}\mathit{\text{across}}\mathit{\text{windows}}\mathit{\text{and}}\mathit{\text{limited}}\mathit{\text{receptive}}\mathit{\text{fields}}$. This highlights the need for more adaptive and flexible attention mechanisms. In this paper, we propose the Deformable Sliding Window Transformer for Image Restoration (DSwinIR), a new attention mechanism: the {Deformable Sliding Window (DSwin) Attention}. {This mechanism introduces a token-centric and content-aware paradigm that moves beyond the grid and fixed window partition.} It comprises two complementary components. First, it replaces the rigid partitioning with a $\mathit{\text{token-centric}}\mathit{\text{sliding}}\mathit{\text{window}}$ paradigm, {making it effective at eliminating boundary artifacts}. Second, it incorporates a $\mathit{\text{content-aware}}\mathit{\text{deformable}}\mathit{\text{sampling}}$ strategy, which allows the attention mechanism to learn data-dependent offsets and actively shape its receptive field to focus on the most informative image regions. Extensive experiments show that DSwinIR achieves strong results, including state-of-the-art performance on several evaluated benchmarks. For instance, in all-in-one image restoration, our DSwinIR surpasses the most recent backbone GridFormer by 0.53 dB on the three-task benchmark and 0.87 dB on the five-task benchmark.