Time-Frequency Analysis for Neural Networks

We develop a quantitative approximation theory for shallow neural networks using tools from time-frequency analysis. Working in weighted modulation spaces $M_m^{p,q}({\mathbf{R}}^d)$, we prove dimension-independent approximation rates in Sobolev norms $W^{n,r}(\mathrm{\Omega })$ for networks whose units combine standard activations with localized time-frequency windows. Our main result shows that for $f\in M_m^{p,q}({\mathbf{R}}^d)$ one can achieve $$\mathrm{\parallel }f-f_N{\mathrm{\parallel }}_{W^{n,r}(\mathrm{\Omega })}\lesssim N^{-1\mathrm{/}2}\mathrm{\parallel }f{\mathrm{\parallel }}_{M_m^{p,q}({\mathbf{R}}^d)},$$ on bounded domains, with explicit control of all constants. We further obtain global approximation theorems on ${\mathbf{R}}^d$ using weighted modulation dictionaries, and derive consequences for Feichtinger’s algebra, Fourier-Lebesgue spaces, and Barron spaces. Numerical experiments in one and two dimensions confirm that modulation-based networks achieve substantially better Sobolev approximation than standard ReLU networks, consistent with the theoretical estimates.