$p$-Groups for which each outer $p$-automorphism centralizes only $p$ elements

An automorphism of a group is called outer if it is not an inner automorphism. Let $G$ be a finite $p$-group. Then for every outer $p$-automorphism $\varphi$ of $G$ the subgroup $C_G(\varphi )=\{x\in G\mathrm{\mid }{x}^{\varphi }=x\}$ has order $p$ if and only if $G$ is of order at most $p^2$.