Foreground Removal in Ground-Based CMB Observations Using a Transformer Model

We present a novel method for Cosmic Microwave Background (CMB) foreground removal based on deep learning techniques. This method employs a Transformer model, referred to as $\mathtt{\text{TCMB}}$, which is specifically designed to effectively process HEALPix-format spherical sky maps. $\mathtt{\text{TCMB}}$ represents an innovative application in CMB data analysis, as it is an image-based technique that has rarely been utilized in this field. Using simulated data with noise levels representative of current ground-based CMB polarization observations, the $\mathtt{\text{TCMB}}$ method demonstrates robust performance in removing foreground contamination. The mean absolute variance for the reconstruction of the noisy CMB Q/U map is significantly less than the CMB polarization signal. To mitigate biases caused by instrumental noise, a cross-correlation approach using two half-mission maps was employed, successfully recovering CMB EE and BB power spectra that align closely with the true values, and these results validate the effectiveness of the $\mathtt{\text{TCMB}}$ method. Compared to the previously employed convolutional neural network (CNN)-based approach, the $\mathtt{\text{TCMB}}$ method offers two significant advantages: (1) It demonstrates superior effectiveness in reconstructing CMB polarization maps, outperforming CNN-based methods. (2) It can directly process HEALPix spherical sky maps without requiring rectangular region division, a step necessary for CNN-based approaches that often introduces uncertainties such as boundary effects. This study highlights the potential of Transformer-based models as a powerful tool for CMB data analysis, offering a substantial improvement over traditional CNN-based techniques.