Ergodic Theorems for discrete Markov chains

Let $X_n$ be a discrete time Markov chain with state space $S$ (countably infinite, in general) and initial probability distribution $\mu^{(0)} = (P(X_0=i_1),P(X_0=i_2),\cdots,)$. What is the probability of choosing in random some $k\in \mathbb{N}$ with $k\le n$ such that $X_k=j$ where $j\in S$? This probability is the average $\frac{1}{n}{\sum }_{k=1}^n{\mu }_j^{(k)}$ where ${\mu }_j^{(k)}=P(X_k=j)$. In this note we will study the limit of this average without assuming that the chain is irreducible, using elementary mathematical tools. Finally, we study the limit of the average $\frac{1}{n} \sum_{k=1}^n g(X_k)$where$g$ is a given function for a Markov chain not necessarily irreducible.