All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtesting

Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level evaluation framework that decomposes prediction rationales into atomic claims and applies Shapley values to quantify each claim’s decision impact, yielding $\mathbf{\text{Shapley-DCLR}}$ ($\mathbf{\text{Shapley}}$-weighted $\mathbf{\text{D}}$ecision-$\mathbf{\text{C}}$ritical $\mathbf{\text{L}}$eakage $\mathbf{\text{R}}$ate) – an interpretable metric measuring what fraction of decision-driving reasoning is contaminated. We further propose $\mathbf{\text{TimeSPEC}}$ ($\mathbf{\text{Time}}$-$\mathbf{\text{S}}$upervised $\mathbf{\text{P}}$rediction with $\mathbf{\text{E}}$xtracted $\mathbf{\text{C}}$laims), an inference-time architecture that interleaves temporally-filtered retrieval with claim-level supervision, producing predictions grounded entirely in pre-cutoff evidence. Across three LLMs, the ablation experiments confirm retrieval and supervision are jointly necessary; and a three-task probe further illstrates that the performance cost of temporal enforcement scales with each task’s reliance on post-cutoff information.