Schoenberg characterization of continuous non-stationary isotropic positive definite kernels

We provide a characterization for the continuous positive definite kernels on ${\mathbb{R}}^d$ that are invariant to linear isometries, i.e. invariant under the orthogonal group $O(d)$. Furthermore, we provide necessary and sufficient conditions for these kernels to be strictly positive definite. This class of isotropic kernels is fairly general: First, it unifies stationary isotropic and dot product kernels, and second, it includes neural network kernels that arise from infinite-width limits of neural networks.