ReGLA: Efficient Receptive-Field Modeling with Gated Linear Attention Network

Balancing accuracy and latency on high-resolution images is a critical challenge for lightweight models, particularly for Transformer-based architectures that often suffer from excessive latency. To address this issue, we introduce $\mathbf{\text{ReGLA}}$, a series of lightweight hybrid networks, which integrates efficient convolutions for local feature extraction with ReLU-based gated linear attention for global modeling. The design incorporates three key innovations: the Efficient Large Receptive Field (ELRF) module for enhancing convolutional efficiency while preserving a large receptive field; the ReLU Gated Modulated Attention (RGMA) module for maintaining linear complexity while enhancing local feature representation; and a multi-teacher distillation strategy to boost performance on downstream tasks. Extensive experiments validate the superiority of ReGLA; particularly the ReGLA-M achieves $\mathbf{\text{80.85\%}}$ Top-1 accuracy on ImageNet-1K at $224px$, with only $\mathbf{\text{4.98}}\mathbf{\text{ms}}$ latency at $512px$. Furthermore, ReGLA outperforms similarly scaled iFormer models in downstream tasks, achieving gains of $\mathbf{\text{3.1\%}}$ AP on COCO object detection and $\mathbf{\text{3.6\%}}$ mIoU on ADE20K semantic segmentation, establishing it as a state-of-the-art solution for high-resolution visual applications.