Permanents of $3\times 3$ Invertible Matrices Modulo $n$

We count the number of elements in the set $$G_3(n,x)=\{M\in G{L}_3({\mathbb{Z}}_n)\mid perm(M)\equiv x(\mathrm{m}\mathrm{o}\mathrm{d}n)\}\mathrm{.}$$