Rethinking UMM Visual Generation: Masked Modeling for Efficient Image-Only Pre-training

Unified Multimodal Models (UMMs) are often constrained by the pre-training of their $\mathbf{\text{visual}}\mathbf{\text{generation}}\mathbf{\text{components}}$, which typically relies on inefficient paradigms and scarce, high-quality text-image paired data. In this paper, we systematically analyze pre-training recipes for $\mathbf{\text{UMM}}\mathbf{\text{visual}}\mathbf{\text{generation}}$ and identify these two issues as the major bottlenecks. To address them, we propose $\mathbf{\text{Image-Only}}\mathbf{\text{Training}}\mathbf{\text{for}}\mathbf{\text{UMMs}}\mathbf{\text{(IOMM)}}$, a data-efficient two-stage training framework. The first stage pre-trains the visual generative component $\mathbf{\text{exclusively}}$ using abundant unlabeled image-only data, thereby removing the dependency on paired data $\mathbf{\text{for}}\mathbf{\text{this}}\mathbf{\text{costly}}\mathbf{\text{phase}}$. The second stage fine-tunes the model using a mixture of unlabeled images and a small curated set of text-image pairs, leading to improved instruction alignment and generative quality. Extensive experiments show that IOMM not only improves training efficiency but also achieves state-of-the-art (SOTA) performance. For example, our IOMM-B (3.6B) model was trained from scratch using only $\sim \mathbf{\text{1050}}$ H800 GPU hours (with the vast majority, $\mathbf{\text{1000}}$ hours, dedicated to the efficient $\mathbf{\text{image-only}}\mathbf{\text{pre-training}}\mathbf{\text{stage}}$). It achieves $\mathbf{\text{0.89}}$ on GenEval and $\mathbf{\text{0.55}}$ on WISE–surpassing strong baselines such as BAGEL-7B (0.82 & 0.55) and BLIP3-o-4B (0.84 & 0.50). Code is available $\text{href}$.