Finite $p$-groups of conjugate type $\{1,p^3\}$

We classify finite $p$-groups, upto isoclinism, which have only two conjugacy class sizes $1$ and $p^3$. It turns out that the nilpotency class of such groups is $2$.