Infinite extensions with finitely many CM moduli

We show that there are uncountably many algebraic extensions of $\mathbb{Q}$ containing at most finitely many moduli of CM simple principally polarized abelian varieties of any fixed dimension $g⩾1$, generalizing a result of Hultberg in dimension 1.