Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models

The advancement of Large Reasoning Models (LRMs) has catalyzed a paradigm shift from reactive fast thinking'' text generation to systematic, step-by-step slow thinking’’ reasoning, unlocking state-of-the-art performance in complex mathematical and logical tasks. However, the field faces $\mathit{\text{the}}\mathit{\text{fundamental}}\mathit{\text{gap}}\mathit{\text{between}}\mathit{\text{token-level}}\mathit{\text{behavioral}}\mathit{\text{analysis}}\mathit{\text{and}}\mathit{\text{internal}}\mathit{\text{reasoning}}\mathit{\text{mechanisms,}}\mathit{\text{and}}\mathit{\text{the}}\mathit{\text{instability}}\mathit{\text{of}}\mathit{\text{reinforcement}}\mathit{\text{learning}}\mathit{\text{(RL)}}\mathit{\text{for}}\mathit{\text{reasoning}}\mathit{\text{optimization}}\mathit{\text{relying}}\mathit{\text{on}}\mathit{\text{costly}}\mathit{\text{external}}\mathit{\text{verifiers}}$. We identify and formally define $\mathbf{\text{Entropy-Gradient}}\mathbf{\text{Inversion}}$, a robust negative correlation between token entropy and logit gradients that acts as a definitive geometric fingerprint for LRM reasoning capability. Building on this, we propose $\mathbf{\text{Correlation-Regularized}}\mathbf{\text{Group}}\mathbf{\text{Policy}}\mathbf{\text{Optimization}}\mathbf{\text{(CorR-PO)}}$, which embeds this inversion signature into RL reward regularization. Extensive experiments on various reasoning benchmarks across multiple model scales show CorR-PO consistently outperforms state-of-the-art baselines, confirming that stronger inversion directly correlates with superior reasoning performance.