Bessel Function Analysis of Nesterov's ODE in $N$-Player Quadratic Games

We analyze Nesterov’s accelerated gradient descent (NAGD) for Nash equilibrium seeking in $N$-player quadratic games. While the continuous-time NAGD dynamics – the Su–Boyd–Candès ODE – are well understood for convex optimization, their behavior with non-symmetric pseudo-gradient matrices arising in games has not been characterized precisely. We establish spectral characterizations via Bessel function modal analysis: the equilibrium is unstable whenever any eigenvalue of the pseudo-gradient matrix $G$ lies outside ${\mathbb{R}}_{\ge 0}$, and all trajectories converge when every eigenvalue lies in ${\mathbb{R}}_{\ge 0}$ and $G$ is diagonalizable. Remarkably, complex eigenvalues with positive real parts, which ensure stability for first-order gradient dynamics, induce exponential instability in NAGD. This reveals that the momentum mechanism enabling $O(1\mathrm{/}{t}^2)$ convergence in optimization can be detrimental for equilibrium seeking in non-potential games.