QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding

Visual quality assessment (VQA) is increasingly shifting from scalar score prediction toward interpretable quality understanding – a paradigm that demands $\mathit{\text{fine-grained}}\mathit{\text{spatiotemporal}}\mathit{\text{perception}}$ and $\mathit{\text{auxiliary}}\mathit{\text{contextual}}\mathit{\text{information}}$. Current approaches rely on supervised fine-tuning or reinforcement learning on curated instruction datasets, which involve labor-intensive annotation and are prone to dataset-specific biases. To address these challenges, we propose $\mathbf{\text{QualiRAG}}$, a $\mathit{\text{training-free}}$ $\mathbf{\text{R}}$etrieval-$\mathbf{\text{A}}$ugmented $\mathbf{\text{G}}$eneration $\mathbf{\text{(RAG)}}$ framework that systematically leverages the latent perceptual knowledge of large multimodal models (LMMs) for visual quality perception. Unlike conventional RAG that retrieves from static corpora, QualiRAG dynamically generates auxiliary knowledge by decomposing questions into structured requests and constructing four complementary knowledge sources: $\mathit{\text{visual}}\mathit{\text{metadata}}$, $\mathit{\text{subject}}\mathit{\text{localization}}$, $\mathit{\text{global}}\mathit{\text{quality}}\mathit{\text{summaries}}$, and $\mathit{\text{local}}\mathit{\text{quality}}\mathit{\text{descriptions}}$, followed by relevance-aware retrieval for evidence-grounded reasoning. Extensive experiments show that QualiRAG achieves substantial improvements over open-source general-purpose LMMs and VQA-finetuned LMMs on visual quality understanding tasks, and delivers competitive performance on visual quality comparison tasks, demonstrating robust quality assessment capabilities without any task-specific training. The code will be publicly available at https://github.com/clh124/QualiRAG.