On the fibers and semi-algebraicity of ReLU neuromanifolds

We study the semi-algebraicity of the neuromanifold ${\mathcal{M}}_{\mathbf{d}}$ of a feedforward ReLU neural network and its symmetries. We prove that ${\mathcal{M}}_{\mathbf{d}}$ is not a semi-algebraic quotient of the space of weights of the network. We introduce and study the notion of \emph{honest} open subset of the space of weights, where the network does not show any hidden symmetries. Finally, we conjecture that the maximal honest open is always semi-algebraic and prove that in the shallow case it is even Zariski.