The Arbitrary-Placement Problem in Entropy-Minimizing Selection, and a Residual-Entropy Formulation

Entropy-based selection objectives suffer from a fundamental degeneracy: minimizing Shannon entropy $H(p_A)$ rewards confident selection regardless of whether the selected candidate is informative. We address this limitation with the residual entropy $D=H(p_A)-H(p_{\beta })$, where $p_{\beta }$ is induced by candidate trust weights. We prove the exact identity $D=-\mathrm{K}\mathrm{L}(p_A\mathrm{\parallel }{p}_{\beta })-\mathrm{\Delta }$, where $\mathrm{\Delta }$ measures whether the score-induced distribution and trust profile favor the same candidates. Boundary cases establish basic safety: under uniform trust, $D\le 0$ automatically, so an equal-trust, non-starving state is never penalized, while at any one-hot limit, $D\to 0$ regardless of the selected candidate. For the intermediate regime where selection occurs, we prove that $D\le 0$ when candidate ordering by trust agrees pairwise with ordering by informativeness, and derive a tighter certificate based on the leading candidate’s margin over its competitors. These results are independent of the candidate-scoring function and apply to both stationary and dynamically changing information. Experiments with a gradient-based mixture-of-experts router confirm that the ordering conditions can hold during real optimization and show that correct ordering improves downstream performance when candidates are non-interchangeable and selections are used directly rather than averaged. Beyond routing, margin-based reweighting matches or outperforms fixed-strength baselines in a class-imbalance task, while informative selection in a production video-prediction system reduces MSE by approximately 20$\mathrm{\%}$ and transfers to a related species. Residual entropy, therefore, provides a safety criterion for selection and a usable signal for deciding when that selection is informative.