A computational approach to extreme values and related hitting probabilities in level-dependent quasi-birth-death processes

This paper analyzes the dynamics of a level-dependent quasi-birth-death process $\mathcal{X}=\{(I(t),J(t)):t\ge 0\}$, i.e., a bi-variate Markov chain defined on the countable state space ${\cup }_{i=0}^{\mathrm{\infty }}l(i)$ with $l(i)=\{(i,j):j\in \{0,\mathrm{.}\mathrm{.}\mathrm{.},M_i\}\}$, for integers $M_i\in {\mathbb{N}}_0$ and $i\in {\mathbb{N}}_0$, which has the special property that its $q$-matrix has a block-tridiagonal form. Under the assumption that the first passage to the subset $l(0)$ occurs in a finite time with certainty, we characterize the probability law of $({\tau }_{\max },I_{\max },J({\tau }_{\max }))$, where $I_{\max }$ is the running maximum level attained by process $\mathcal{X}$ before its first visit to states in $l(0)$, ${\tau }_{\max }$ is the first time that the level process $\{I(t):t\ge 0\}$ reaches the running maximum $I_{\max }$, and $J({\tau }_{\max })$ is the phase at time ${\tau }_{\max }$. Our methods rely on the use of restricted Laplace-Stieltjes transforms of ${\tau }_{\max }$ on the set of sample paths $\{I_{\max }=i,J({\tau }_{\max })=j\}$, and related processes under taboo of certain subsets of states. The utility of the resulting computational algorithms is demonstrated in two epidemic models: the SIS model for horizontally and vertically transmitted diseases; and the SIR model with constant population size.