Restricted Stirling and Lah number matrices and their inverses

Given $R\subseteq \mathbb{N}$ let ${\left\{\genfrac{}{}{0px}{}{n}{k}\right\}}_R$, ${\left[\genfrac{}{}{0px}{}{n}{k}\right]}_R$, and $L(n,k{)}_R$ be the number of ways of partitioning the set $[n]$ into $k$ non-empty subsets, cycles and lists, respectively, with each block having cardinality in $R$. We refer to these as the $R$-restricted Stirling numbers of the second and first kind and the $R$-restricted Lah numbers, respectively. Note that the classical Stirling numbers of the second kind and first kind, and Lah numbers are $\left\{\genfrac{}{}{0px}{}{n}{k}\right\}={\left\{\genfrac{}{}{0px}{}{n}{k}\right\}}_{\mathbb{N}}$, ${n \brack k} = {n \brack k}{$\mathbb{N}$} $and$L(n,k) = L(n,k){$\mathbb{N}$}$, respectively. The matrices $[\left\{\genfrac{}{}{0px}{}{n}{k}\right\}{]}_{n,k\ge 1}$, $[\left[\genfrac{}{}{0px}{}{n}{k}\right]{]}_{n,k\ge 1}$ and $[L(n,k){]}_{n,k\ge 1}$ have inverses $[(-1{)}^{n-k}\left[\genfrac{}{}{0px}{}{n}{k}\right]{]}_{n,k\ge 1}$, $[(-1{)}^{n-k}\left\{\genfrac{}{}{0px}{}{n}{k}\right\}{]}_{n,k\ge 1}$ and $[(-1{)}^{n-k}L(n,k){]}_{n,k\ge 1}$ respectively. The inverse matrices $[{\left\{\genfrac{}{}{0px}{}{n}{k}\right\}}_R{]}_{n,k\ge 1}^{-1}$, $[{n \brack k}R]^{-1}{n,k \geq 1}$and$[L(n,k)R]^{-1}{n,k \geq 1}$ exist if and only if $1\in R$. We express each entry of each of these matrices as the difference between the cardinalities of two explicitly defined families of labeled forests. In particular the entries of $[{n \brace k}{[r]}]^{-1}{n,k \geq 1}$ have combinatorial interpretations, affirmatively answering a question of Choi, Long, Ng and Smith from 2006. If $1,2\in R$ and if for all $n\in R$ with $n$ odd and $n\ge 3$, we have $n\pm 1\in R$, we additionally show that each entry of $[{n \brace k}R]^{-1}{n,k \geq 1}$,$[{n \brack k}R]^{-1}{n,k \geq 1}$and$[L(n,k)R]^{-1}{n,k \geq 1}$ is up to an explicit sign the cardinality of a single explicitly defined family of labeled forests. Our results also provide combinatorial interpretations of the $k$th Whitney numbers of the first and second kinds of ${\mathrm{\Pi }}_n^{1,d}$, the poset of partitions of $[n]$ that have each part size congruent to $1$ mod $d$.