Deep ReLU network approximation of functions on a manifold

Whereas recovery of the manifold from data is a well-studied topic, approximation rates for functions defined on manifolds are less known. In this work, we study a regression problem with inputs on a $d^{\ast }$-dimensional manifold that is embedded into a space with potentially much larger ambient dimension. It is shown that sparsely connected deep ReLU networks can approximate a H"older function with smoothness index $\beta$ up to error $ϵ$ using of the order of ${ϵ}^{-d^{\ast }\mathrm{/}\beta }\log (1\mathrm{/}ϵ)$ many non-zero network parameters. As an application, we derive statistical convergence rates for the estimator minimizing the empirical risk over all possible choices of bounded network parameters.