On the Fundamental Limit of the Stochastic Gradient Identification Algorithm Under Non-Persistent Excitation

Stochastic gradient (SG) methods are fundamental to system identification and machine learning, enabling online parameter estimation in large-scale and streaming-data settings. As a classical identification method, the SG algorithm has been extensively studied for decades. Under non-persistent excitation, the strongest currently available convergence result assumes that the condition number of the Fisher information matrix is $O((\log r_n{)}^{\alpha })$, where $r_n=1+{\sum }_{i=1}^n\mathrm{\parallel }{\phi }_i{\mathrm{\parallel }}^2$. Existing theory establishes strong consistency when $\alpha \le 1\mathrm{/}3$, whereas the same condition with $\alpha >1$ is insufficient to guarantee strong consistency. We prove that strong consistency holds throughout the range $0\le \alpha <1$. The proof is based on a new algebraic framework that yields substantially sharper matrix norm bounds. This result nearly resolves the four-decade-old Chen–Guo conjecture by establishing strong consistency throughout the previously open range $1\mathrm{/}3<\alpha <1$.