Weighted information and entropy rates

The weighted entropy $H_{\varphi }^{\mathrm{w}}(X)=H_{\varphi }^{\mathrm{w}}(f)$ of a random variable $X$ with values $x$ and a probability-mass/density function $f$ is defined as the mean value $\mathbb{E}{I}_{\varphi }^{\mathrm{w}}(X)$ of the weighted information $I_{\varphi }^{\mathrm{w}}(x)=-\varphi (x)\log f(x)$. Here $x\mapsto\phi (x)\in{\mathbb R}$ is a given weight function (WF) indicating a ‘value’ of outcome $x$. For an $n$-component random vector ${\mathbf{X}}_0^{n-1}=(X_0,\dots ,X_{n-1})$ produced by a random process $\mathbf{X}=(X_i,i\in \mathbb{Z})$, the weighted information $I^{\rm w}_{\phi_n}({\mathbf x}0^{n-1})$andweightedentropy$H^{\rm w}{\phi_n}({$\mathbf{X}$}_0^{n-1})$aredefinedsimilarly,withanWF$\phi_n({\mathbf x}_0^{n-1})$\mathrm{.}TwotypesofWFs$\phi_n$ are considered, based on additive and a multiplicative forms ($\phi_n({\mathbf x}0^{n-1})=\sum\limits{i=0}^{n-1}{\varphi} (x_i)$and$\phi_n({\mathbf x}0^{n-1})=\prod\limits{i=0}^{n-1}{\varphi} (x_i)$, respectively). The focus is upon $rates$ of the weighted entropy and information, regarded as parameters related to $\mathbf{X}$. We show that, in the context of ergodicity, a natural scale for an asymptotically additive/multiplicative WF is $\frac{1}{n^2}{H}_{{\varphi }_n}^{\mathrm{w}}({\mathbf{X}}_0^{n-1})$ and $\frac{1}{n}\log H_{{\varphi }_n}^{\mathrm{w}}({\mathbf{X}}_0^{n-1})$, respectively. This gives rise to $primary$ $rates$. The next-order terms can also be identified, leading to $secondary$ $rates$. We also consider emerging generalisations of the Shannon-McMillan-Breiman theorem.