Pay Attention to Sequence Split: Uncovering the Impacts of Sub-Sequence Splitting on Sequential Recommendation Models

Sub-sequence splitting (SSS) has been demonstrated as an effective approach to mitigate data sparsity in sequential recommendation (SR) by splitting a raw user interaction sequence into multiple sub-sequences. Previous studies have demonstrated its ability to enhance the performance of SR models significantly. However, in this work, we discover that $\mathbf{\text{(i).}}\mathbf{\text{SSS}}\mathbf{\text{may}}\mathbf{\text{interfere}}\mathbf{\text{with}}\mathbf{\text{the}}\mathbf{\text{evaluation}}\mathbf{\text{of}}\mathbf{\text{the}}\mathbf{\text{model’s}}\mathbf{\text{actual}}\mathbf{\text{performance.}}$ We observed that many recent state-of-the-art SR models employ SSS during the data reading stage (not mentioned in the papers). When we removed this operation, performance significantly declined, even falling below that of earlier classical SR models. The varying improvements achieved by SSS and different splitting methods across different models prompt us to analyze further when SSS proves effective. We find that $\mathbf{\text{(ii).}}\mathbf{\text{SSS}}\mathbf{\text{demonstrates}}\mathbf{\text{strong}}\mathbf{\text{capabilities}}\mathbf{\text{only}}\mathbf{\text{when}}\mathbf{\text{specific}}\mathbf{\text{splitting}}\mathbf{\text{methods,}}\mathbf{\text{target}}\mathbf{\text{strategies,}}\mathbf{\text{and}}\mathbf{\text{loss}}\mathbf{\text{functions}}\mathbf{\text{are}}\mathbf{\text{used}}\mathbf{\text{together.}}$ Inappropriate combinations may even harm performance. Furthermore, we analyze why sub-sequence splitting yields such remarkable performance gains and find that $\mathbf{\text{(iii).}}\mathbf{\text{it}}\mathbf{\text{evens}}\mathbf{\text{out}}\mathbf{\text{the}}\mathbf{\text{distribution}}\mathbf{\text{of}}\mathbf{\text{training}}\mathbf{\text{data}}\mathbf{\text{while}}\mathbf{\text{increasing}}\mathbf{\text{the}}\mathbf{\text{likelihood}}\mathbf{\text{that}}\mathbf{\text{different}}\mathbf{\text{items}}\mathbf{\text{are}}\mathbf{\text{targeted.}}$ Finally, we provide suggestions for overcoming SSS interference, along with a discussion on data augmentation methods and future directions. We hope this work will prompt the broader community to re-examine the impact of data splitting on SR and promote fairer, more rigorous model evaluation. All analysis code and data will be made available upon acceptance. We provide a simple, anonymous implementation at https://github.com/KingGugu/SSS4SR.