A Matrix Gaussian Distribution

In this note, we define a Gaussian probability distribution over matrices. We prove some useful properties of this distribution, namely, the fact that marginalization, conditioning, and affine transformations preserve the matrix Gaussian distribution. We also derive useful results regarding the expected value of certain quadratic forms based solely on covariances between matrices. Previous definitions of matrix normal distributions are severely under-parameterized, assuming unrealistic structure on the covariance (see Section 2). We believe that our generalization is better equipped for use in practice.