On a question of K\"ulshammer for homomorphisms of algebraic groups

Let $G$ be a linear algebraic group over an algebraically closed field of characteristic $p\ge 0$. We show that if $H_1$ and $H_2$ are connected subgroups of $G$ such that $H_1$ and $H_2$ have a common maximal unipotent subgroup and $H_1\mathrm{/}{R}_u(H_1)$ and $H_2\mathrm{/}{R}_u(H_2)$ are semisimple, then $H_1$ and $H_2$ are $G$-conjugate. Moreover, we show that if $H$ is a semisimple linear algebraic group with maximal unipotent subgroup $U$ then for any algebraic group homomorphism $\sigma \text{\hspace{0.17em}⁣}:U\to G$, there are only finitely many $G$-conjugacy classes of algebraic group homomorphisms $\rho\colon H\rightarrow G$suchthat$\rho|_U$is$G$-conjugateto$\sigma$. This answers an analogue for connected algebraic groups of a question of B. K"ulshammer. In K"ulshammer’s original question, $H$ is replaced by a finite group and $U$ by a Sylow $p$-subgroup of $H$; the answer is then known to be no in general. We obtain some results in the general case when $H$ is non-connected and has positive dimension. Along the way, we prove existence and conjugacy results for maximal unipotent subgroups of non-connected linear algebraic groups. When $G$ is reductive, we formulate K"ulshammer’s question and related conjugacy problems in terms of the nonabelian 1-cohomology of unipotent radicals of parabolic subgroups of $G$, and we give some applications of this cohomological approach. In particular, we analyse the case when $G$ is a semisimple group of rank 2.