The field of moduli of sets of points in ${\mathbb{P}}^2$

For every $n\ge 6$, we give an example of a finite subset of ${\mathbb{P}}^2$ of degree $n$ which does not descend to any Brauer-Severi surface over the field of moduli. Conversely, for every $n\le 5$ we prove that a finite subset of degree $n$ always descends to a $0$-cycle on ${\mathbb{P}}^2$ over the field of moduli.