Sharp uniform approximation for spectral Barron functions by deep neural networks

This work explores the neural network approximation capabilities for functions within the spectral Barron space ${\mathcal{B}}^s$, where $s$ is the smoothness index. We demonstrate that for functions in ${\mathcal{B}}^{1\mathrm{/}2}$, a shallow neural network (a single hidden layer) with $N$ units can achieve an $L^p$-approximation rate of $\mathcal{O}(N^{-1\mathrm{/}2})$. This rate also applies to uniform approximation, differing by at most a logarithmic factor. Our results significantly reduce the smoothness requirement compared to existing theory, which necessitate functions to belong to ${\mathcal{B}}^1$ in order to attain the same rate. Furthermore, we show that increasing the network’s depth can notably improve the approximation order for functions with small smoothness. Specifically, for networks with $L$ hidden layers, functions in ${\mathcal{B}}^s$ with $0<sL\le 1\mathrm{/}2$ can achieve an approximation rate of $\mathcal{O}(N^{-sL})$. The rates and prefactors in our estimates are dimension-free. We also confirm the sharpness of our findings, with the lower bound closely aligning with the upper, with a discrepancy of at most one logarithmic factor.