CroBIM-U: Uncertainty-Driven Referring Remote Sensing Image Segmentation

Referring remote sensing image segmentation aims to localize specific targets described by natural language within complex overhead imagery. However, due to extreme scale variations, dense similar distractors, and intricate boundary structures, the reliability of cross-modal alignment exhibits significant $\mathbf{\text{spatial}}\mathbf{\text{non-uniformity}}$. Existing methods typically employ uniform fusion and refinement strategies across the entire image, which often introduces unnecessary linguistic perturbations in visually clear regions while failing to provide sufficient disambiguation in confused areas. To address this, we propose an $\mathbf{\text{uncertainty-guided}}\mathbf{\text{framework}}$ that explicitly leverages a pixel-wise $\mathbf{\text{referring}}\mathbf{\text{uncertainty}}\mathbf{\text{map}}$ as a spatial prior to orchestrate adaptive inference. Specifically, we introduce a plug-and-play $\mathbf{\text{Referring}}\mathbf{\text{Uncertainty}}\mathbf{\text{Scorer}}\mathbf{\text{(RUS)}}$, which is trained via an online error-consistency supervision strategy to interpretably predict the spatial distribution of referential ambiguity. Building on this prior, we design two plug-and-play modules: 1) $\mathbf{\text{Uncertainty-Gated}}\mathbf{\text{Fusion}}\mathbf{\text{(UGF)}}$, which dynamically modulates language injection strength to enhance constraints in high-uncertainty regions while suppressing noise in low-uncertainty ones; and 2) $\mathbf{\text{Uncertainty-Driven}}\mathbf{\text{Local}}\mathbf{\text{Refinement}}\mathbf{\text{(UDLR)}}$, which utilizes uncertainty-derived soft masks to focus refinement on error-prone boundaries and fine details. Extensive experiments demonstrate that our method functions as a unified, plug-and-play solution that significantly improves robustness and geometric fidelity in complex remote sensing scenes without altering the backbone architecture.