On Horn's Problem and its Volume Function

We consider an extended version of Horn’s problem: given two orbits ${\mathcal{O}}_{\alpha }$ and ${\mathcal{O}}_{\beta }$ of a linear representation of a compact Lie group, let $A\in {\mathcal{O}}_{\alpha }$, $B\in {\mathcal{O}}_{\beta }$ be independent and invariantly distributed random elements of the two orbits. The problem is to describe the probability distribution of the orbit of the sum $A+B$. We study in particular the familiar case of coadjoint orbits, and also the orbits of self-adjoint real, complex and quaternionic matrices under the conjugation actions of $\mathrm{S}\mathrm{O}(n)$, $\mathrm{S}\mathrm{U}(n)$ and $\mathrm{U}\mathrm{S}\mathrm{p}(n)$ respectively. The probability density can be expressed in terms of a function that we call the volume function. In this paper, (i) we relate this function to the symplectic or Riemannian geometry of the orbits, depending on the case; (ii) we discuss its non-analyticities and possible vanishing; (iii) in the coadjoint case, we study its relation to tensor product multiplicities (generalized Littlewood–Richardson coefficients) and show that it computes the volume of a family of convex polytopes introduced by Berenstein and Zelevinsky. These considerations are illustrated by a detailed study of the volume function for the coadjoint orbits of $B_2=\mathfrak{s}\mathfrak{o}(5)$.