A Tight Analysis of Hutchinson's Diagonal Estimator

Let $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ be a matrix with diagonal $\text{diag}(\mathbf{A})$ and let $\bar{\mathbf{A}}$ be $\mathbf{A}$ with its diagonal set to all zeros. We show that Hutchinson’s estimator run for $m$ iterations returns a diagonal estimate $\tilde{d}\in {\mathbb{R}}^n$ such that with probability $(1-\delta )$, $$\mathrm{\parallel }\tilde{d}-\text{diag}(\mathbf{A}){\mathrm{\parallel }}_2\le c\sqrt{\frac{\log (2\mathrm{/}\delta )}{m}}\mathrm{\parallel }\bar{\mathbf{A}}{\mathrm{\parallel }}_F,$$ where $c$ is a fixed constant independent of all other parameters. This results improves on a recent result of [Baston and Nakatsukasa, 2022] by a $\log (n)$ factor, yielding a bound that is independent of the matrix dimension $n$.