The Complexity of Finding Stationary Points in Nonsmooth Nonconvex Optimization

We prove that first-order algorithms require $\mathrm{\Omega }({\delta }^{-1}{\epsilon }^{-3})$ gradient queries (in the worst case) to find a $(\delta ,\epsilon )$-Goldstein stationary point of a Lipschitz function, at which there is a convex combination of gradients within distance $\delta$ whose norm is at most $\epsilon$. This lower bound is tight, matching known algorithms up to absolute constants, therefore resolving the complexity of convergence to stationarity in nonsmooth nonconvex optimization. We further prove a tight lower bound of $\mathrm{\Omega }({\lambda }^{1\mathrm{/}2}{\epsilon }^{-7\mathrm{/}2})$ for finding points satisfying the recently proposed relaxed notion of $(\lambda ,\epsilon )$-stationarity, which allows combining further-away gradients. Our results reveal that convergence rates to nonsmooth stationarity are not affected by gradient stochasticity, in sharp contrast to smooth optimization.