Approximation Rates for Shallow ReLU${}^k$ Neural Networks on Sobolev Spaces via the Radon Transform

Let $\mathrm{\Omega }\subset {\mathbb{R}}^d$ be a bounded domain. We consider the problem of how efficiently shallow neural networks with the ReLU${}^k$ activation function can approximate functions from Sobolev spaces $W^s(L_p(\mathrm{\Omega }))$ with error measured in the $L_q(\mathrm{\Omega })$-norm. Utilizing the Radon transform and recent results from discrepancy theory, we provide a simple proof of nearly optimal approximation rates in a variety of cases, including when $q\le p$, $p\ge 2$, and $s\le k+(d+1)\mathrm{/}2$. The rates we derive are optimal up to logarithmic factors, and significantly generalize existing results. An interesting consequence is that the adaptivity of shallow ReLU${}^k$ neural networks enables them to obtain optimal approximation rates for smoothness up to order $s=k+(d+1)\mathrm{/}2$, even though they represent piecewise polynomials of fixed degree $k$.