Physics informed learning of orthogonal features with applications in solving partial differential equations

The random feature method (RFM) constructs approximation spaces by initializing features from generic distributions, which provides universal approximation properties to solve general partial differential equations. However, such standard initializations lack awareness of the underlying physical laws and geometry, which limits approximation. In this work, we propose the Physics-Driven Orthogonal Feature Method (PD-OFM), a framework for constructing feature representations that are explicitly tailored to both the differential operator and the computational domain by pretraining features using physics-informed objectives together with orthogonality regularization. This pretraining strategy yields nearly orthogonal feature bases. We provide both theoretical and empirical evidence that physics-informed pretraining improves the approximation capability of the learned feature space. When employed to solve Helmholtz, Poisson, wave, and Navier-Stokes equations, the proposed method achieves residual errors 2-3 orders of magnitude lower than those of comparable methods. Furthermore, the orthogonality regularization improves transferability, enabling pretrained features to generalize effectively across different source terms and domain geometries for the same PDE.