Minimal-order groups with an irreducible character of degree $p$ or $p^2$

We characterize the finite groups of minimal order that admit an irreducible complex character of degree $p$ or $p^2$, where $p$ is a prime.