Not Only NTP: Extending Training Signal Coverage for Generative Recommendation

Next-Token Prediction (NTP) carries two structural training signal limitations. First, NTP optimizes for single-step prediction only, placing no supervised pressure on learning longer-range behavioral structure – we term this $\mathbf{\text{temporal}}\mathbf{\text{locality}}$. Second, in multi-domain sequences, each target item embedding receives gradient updates exclusively from the immediately preceding hidden state, with no explicit gradient pathway from cross-domain context – we term this $\mathbf{\text{spatial}}\mathbf{\text{locality}}$. We propose $\mathbf{\text{NONTP}}$, extending NTP’s signal coverage along both dimensions through two auxiliary objectives. $\mathbf{\text{TCL}}\mathbf{\text{(Temporal}}\mathbf{\text{Contrastive}}\mathbf{\text{Learning)}}$ uses a BYOL-style EMA teacher with InfoNCE to align hidden states against a $K$-step future trajectory in representation space. $\mathbf{\text{TDL}}\mathbf{\text{(Trans-Domain}}\mathbf{\text{Learning)}}$ mean-pools cross-domain hidden states and predicts through the shared prediction head, opening a second gradient pathway with no additional parameters. Both are discarded at inference: zero overhead. On a four-domain Meituan industrial dataset (full ranking), NONTP achieves HR@10 +34.3% over NTP and +18.3% over MBGR. On the public Amazon Movie-Book-CDs benchmark, HR@10 +2.8% and NDCG@10 +3.7%. Online A/B tests confirm CTR +1.8% and GMV +2.1% (both $p<0.01$). Ablation studies confirm each component contributes independently, with gradient conflict analyzed as a direction for future work.