Sharp heat kernel estimates for relativistic stable processes in open sets

In this paper, we establish sharp two-sided estimates for the transition densities of relativistic stable processes [i.e., for the heat kernels of the operators $m-(m^{2\mathrm{/}\alpha }-\mathrm{\Delta }{)}^{\alpha \mathrm{/}2}$] in $C^{1,1}$ open sets. Here $m>0$ and $\alpha \in (0,2)$. The estimates are uniform in $m\in (0,M]$ for each fixed $M>0$. Letting $m\downarrow 0$, we recover the Dirichlet heat kernel estimates for ${\mathrm{\Delta }}^{\alpha \mathrm{/}2}:=-(-\mathrm{\Delta }{)}^{\alpha \mathrm{/}2}$ in $C^{1,1}$ open sets obtained in [14]. Sharp two-sided estimates are also obtained for Green functions of relativistic stable processes in bounded $C^{1,1}$ open sets.