BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment

Multimodal retrieval systems struggle to resolve image-text queries against text-only corpora: the best vision-language encoder achieves only 27.6 nDCG@10 on MM-BRIGHT, underperforming strong text-only retrievers. We argue the bottleneck is not the retriever but the query – raw multimodal queries entangle visual descriptions, conversational noise, and retrieval intent in ways that systematically degrade embedding similarity. We present $\mathbf{\text{BRIDGE}}$, a two-component system that resolves this mismatch without multimodal encoders. $\mathbf{\text{FORGE}}$ ($\mathbf{\text{F}}$ocused Retrieval Query Generato$\mathbf{\text{r}}$) is a query alignment model trained via reinforcement learning, which distills noisy multimodal queries into compact, retrieval-optimized search strings. $\mathbf{\text{LENS}}$ ($\mathbf{\text{L}}$anguage-$\mathbf{\text{E}}$nhanced $\mathbf{\text{N}}$eural $\mathbf{\text{S}}$earch) is a reasoning-enhanced dense retriever fine-tuned on reasoning-intensive retrieval data to handle the intent-rich queries FORGE produces. Evaluated on MM-BRIGHT (2,803 queries, 29 domains), BRIDGE achieves $\mathbf{\text{29.7}}$ nDCG@10, surpassing all multimodal encoder baselines including Nomic-Vision (27.6). When FORGE is applied as a plug-and-play aligner on top of Nomic-Vision, the combined system reaches $\mathbf{\text{33.3}}$ nDCG@10 – exceeding the best text-only retriever (32.2) – demonstrating that $\mathit{\text{query}}\mathit{\text{alignment}}$ is the key bottleneck in multimodal-to-text retrieval. https://github.com/mm-bright/multimodal-reasoning-retrieval