A Central Limit Theorem for Regularized M-Estimators

We prove a quantitative central limit theorem for linear functionals of regularized empirical-risk minimizers in the proportional-dimensional regime $p=O(n)$. The data columns are independent, not necessarily identically distributed, and satisfy a uniform columnwise Poincaré inequality. Under uniform curvature and smoothness assumptions, and for a quadratic regularizer, we show that every nondegenerate statistic $\sqrt{n}{u}^{\mathrm{\top }}\widehat{\theta }$, centered by its expectation and normalized by its standard deviation, converges to a standard normal random variable in Wasserstein distance, with rate $O((\log n{)}^7{n}^{-1\mathrm{/}4})$. The proof is based on moment and stability bounds for the minimizer, a second-order leave-one-out expansion, and a perturbative normal-approximation argument for functions of independent variables. We also prove the variance upper bound $\text{Var}(u^{\mathrm{\top }}\widehat{\theta })\le C\text{norm}{u}_2^2\mathrm{/}n$, identifying the $\sqrt{n}$ fluctuation scale.