Rarity Determines Minimax Rates of Semantic Entropy From Weak Pairwise Judgments With a Known Channel

The semantic entropy of a language model is the entropy of the meanings of its sampled answers. The meanings are never observed, and the entropy is estimated from the verdicts of a pairwise equivalence judge. Given the meanings of $n$ answers, a known channel declares each pair equivalent independently, with probability $q+\mathrm{\Delta }$ within a meaning and $q$ across meanings. At most $K$ meanings occur, and $q$ is bounded away from $0$ and $1$. For a weak judge, with $n{\mathrm{\Delta }}^2\to 0$, the global minimax rate is set by distributions near a vertex of the simplex and overstates the difficulty elsewhere. We show that the difficulty is set by how many meanings can be rare. Over distributions with at most $s$ meanings below a fixed probability $\tau <1\mathrm{/}(K-s)$, the minimax squared risk is of order $N^{-1\mathrm{/}s}$ for $s\ge 2$, $N^{-1}{\log }^2(e+N)$ for $s=1$ and $N^{-1}$ for $s=0$, capped at one, where $N=n^K{\mathrm{\Delta }}^{2K-2}$. The matching lower bounds hold on every fixed polynomial window $n{\mathrm{\Delta }}^2\le n^{-\kappa }$. One estimator attains these rates without knowing $s$ or $\tau$: it averages products of centered verdicts along all spanning trees of every $k$-set of answers, projects the resulting power sums onto the valid moment set and inverts them. The case $s=K-1$ gives the global rate $N^{-1\mathrm{/}(K-1)}$ and the consistency threshold: with $\mathrm{\Delta }=n^{-\gamma }$ the entropy is uniformly consistently estimable exactly when $\gamma <K\mathrm{/}(2K-2)$. On polynomial windows, the same estimator attains the sharp local minimax constant on shrinking neighborhoods of interior points with distinct probabilities. Restricting the average to stars multiplies the constant by $K^{K-3}$.