A General Derivative Identity for the Conditional Expectation with Focus on the Exponential Family

Consider a pair of random vectors $(\mathbf{X},\mathbf{Y})$ and the conditional expectation operator $\mathbb{E}[\mathbf{X}\mathrm{\mid }\mathbf{Y}=\mathbf{y}]$. This work studies analytic properties of the conditional expectation by characterizing various derivative identities. The paper consists of two parts. In the first part of the paper, a general derivative identity for the conditional expectation is derived. Specifically, for the Markov chain $$\mathbf{U}$ \leftrightarrow $\mathbf{X}$ \leftrightarrow $\mathbf{Y}$$,acompactexpressionfortheJacobianmatrixof$$\mathbb{E}$[$\mathbf{U}$|$\mathbf{Y}$=$\mathbf{y}$]$ is derived. In the second part of the paper, the main identity is specialized to the exponential family. Moreover, via various choices of the random vector $\mathbf{U}$, the new identity is used to recover and generalize several known identities and derive some new ones. As a first example, a connection between the Jacobian of $\mathbb{E}[\mathbf{X}\mathrm{\mid }\mathbf{Y}=\mathbf{y}]$ and the conditional variance is established. As a second example, a recursive expression between higher order conditional expectations is found, which is shown to lead to a generalization of the Tweedy’s identity. Finally, as a third example, it is shown that the $k$-th order derivative of the conditional expectation is proportional to the $(k+1)$-th order conditional cumulant.