NSTR: Neural Spectral Transport Representation for Space-Varying Frequency Fields

Implicit Neural Representations (INRs) have emerged as a powerful paradigm for representing signals such as images, audio, and 3D scenes. However, existing INR frameworks – including MLPs with Fourier features, SIREN, and multiresolution hash grids – implicitly assume a $\mathit{\text{global}}\mathit{\text{and}}\mathit{\text{stationary}}$ spectral basis. This assumption is fundamentally misaligned with real-world signals whose frequency characteristics vary significantly across space, exhibiting local high-frequency textures, smooth regions, and frequency drift phenomena. We propose $\mathbf{\text{Neural}}\mathbf{\text{Spectral}}\mathbf{\text{Transport}}\mathbf{\text{Representation}}\mathbf{\text{(NSTR)}}$, the first INR framework that $\mathbf{\text{explicitly}}\mathbf{\text{models}}\mathbf{\text{a}}\mathbf{\text{spatially}}\mathbf{\text{varying}}\mathbf{\text{local}}\mathbf{\text{frequency}}\mathbf{\text{field}}$. NSTR introduces a learnable \emph{frequency transport equation}, a PDE that governs how local spectral compositions evolve across space. Given a learnable local spectrum field $S(x)$ and a frequency transport network $F_{\theta }$ enforcing $\mathrm{\nabla }S(x)\approx F_{\theta }(x,S(x))$, NSTR reconstructs signals by spatially modulating a compact set of global sinusoidal bases. This formulation enables strong local adaptivity and offers a new level of interpretability via visualizing frequency flows. Experiments on 2D image regression, audio reconstruction, and implicit 3D geometry show that NSTR achieves significantly better accuracy-parameter trade-offs than SIREN, Fourier-feature MLPs, and Instant-NGP. NSTR requires fewer global frequencies, converges faster, and naturally explains signal structure through spectral transport fields. We believe NSTR opens a new direction in INR research by introducing explicit modeling of space-varying spectrum.