Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation

Scaling recommendation models is a central challenge in recommender systems. Recently, RankMixer has emerged as an effective solution, operating on a unified token representation and alternating between token mixing and per-token feedforward networks (P-FFNs) to achieve scalable performance. However, RankMixer suffers from $\mathit{\text{embedding}}\mathit{\text{collapse}}$, where learned representations have low effective rank, limiting expressivity and underutilizing the expanded representation space. Through empirical analysis and theoretical insights, we identify rigid token mixing and P-FFN modules as the primary causes of this phenomenon, jointly inducing a $\mathbf{\text{damped}}\mathbf{\text{oscillatory}}\mathbf{\text{trajectory}}$ in effective-rank evolution across layers. To address it, we propose RankElastor, a novel architecture that produces spectrum-robust representations with provable collapse mitigation. RankElastor introduces two components: (i) $\mathbf{\text{parameterized}}\mathbf{\text{full}}\mathbf{\text{mixing}}$, which enables expressive token mixing with improved spectral robustness; and (ii) $\mathbf{\text{GLU-improved}}\mathbf{\text{P-FFNs}}$, which stabilize representation spectra through GLU-style FFN modules. Extensive experiments on large-scale industrial datasets demonstrate that RankElastor consistently improves recommendation performance, mitigates embedding collapse, and exhibits robust scaling behavior. Code is available at this GitHub repository: https://github.com/vasile-paskardlgm/RankElastor