On the consistency of the Kozachenko-Leonenko entropy estimate

We revisit the problem of the estimation of the differential entropy $H(f)$ of a random vector $X$ in $R^d$ with density $f$, assuming that $H(f)$ exists and is finite. In this note, we study the consistency of the popular nearest neighbor estimate $H_n$ of Kozachenko and Leonenko. Without any smoothness condition we show that the estimate is consistent ($E\{\mathrm{\mid }{H}_n-H(f)\mathrm{\mid }\}\to 0$ as $n\to \mathrm{\infty }$) if and only if $\mathbb{E}\{\log (\mathrm{\parallel }X\mathrm{\parallel }+1)\}<\mathrm{\infty }$. Furthermore, if $X$ has compact support, then $H_n\to H(f)$ almost surely.