The dual group of a spherical variety

Let $X$ be a spherical variety for a connected reductive group $G$. Work of Gaitsgory-Nadler strongly suggests that the Langlands dual group $G^{\vee }$ of $G$ has a subgroup whose Weyl group is the little Weyl group of $X$. Sakellaridis-Venkatesh defined a refined dual group $G_X^{\vee }$ and verified in many cases that there exists an isogeny $\varphi$ from $G_X^{\vee }$ to $G^{\vee }$. In this paper, we establish the existence of $\varphi$ in full generality. Our approach is purely combinatorial and works (despite the title) for arbitrary $G$-varieties.