Self-Supervised Representation Learning for JWST Galaxy Images and Low Resolution Spectra

A self-supervised multimodal model recovers galaxy stellar masses from sparse JWST data that lack the multi-band coverage required by traditional SED fitting. We train on single-band F150W imaging paired with observed-frame 1-2 $\mu$m PRISM spectra and obtain a joint image-spectrum representation that recovers stellar mass at $1<z\le 3$ with $R^2\sim 0.9$ and ${\sigma }_{\mathrm{N}\mathrm{M}\mathrm{A}\mathrm{D}}=0.24$ dex; cross-modal CLIP alignment further raises single-modality performance by $\mathrm{\Delta }{R}^2=+0.04$ (imaging) and $+0.09$ (spectroscopy), narrowing the gap to joint-modality inference. The code and documentation are available at \href{https://github.com/2003cy/neurips2026_ssl_jwst}{GitHub}.