Randomly Permuted Orthogonal Products and Fast Dimension Reduction

We study the effect of random signed permutations on products of orthogonal matrices and their applications to fast dimension reduction. Let $A,B\in {\mathbb{R}}^{d\times d}$ be orthogonal matrices and let $\mathrm{\Sigma }\in {\mathbb{R}}^{d\times d}$ be a uniformly random signed permutation matrix. We analyze the random orthogonal matrix $$U=A\mathrm{\Sigma }B,$$ and show that, under mild assumptions on the size of the entries of $A$ and $B$, $$\underset{i,j=1,\dots ,d}{\max }\mathrm{\mid }{U}_{ij}\mathrm{\mid }=O\left(\sqrt{\frac{\log d}{d}}\right)$$ with high probability. As an application, we show that ORA, an analogue of the Kac walk in which every update is a $\pi \mathrm{/}4$ rotation, reaches the same maximal entry scale after $O(d\log d)$ updates. This resolves a question of Jain et al. and improves the running time of their construction. We also show that parallel ORA reaches this scale after $O(\log d)$ rounds. We then study the random embedding $$\mathrm{\Phi }=\sqrt{\frac{d}{m}}{P}_{I}U{D}_{{\xi }^{\mathrm{\prime }}},$$ where $P_I$ restricts to $m$ coordinates and ${\xi }^{\mathrm{\prime }}$ is an independent Rademacher vector. We identify two parameters controlling norm preservation and show that, throughout the corresponding admissible range, $\mathrm{\Phi }$ achieves optimal embedding dimension $m\asymp {\epsilon }^{-2}\log (N)$. Finally, we extend the result to structured infinite models, including sparse vectors, low-rank matrices, and finite unions of subspaces.