Autoencoding Dynamics: Topological Limitations and Capabilities

Given a “data manifold” $M\subset {\mathbb{R}}^n$ and “latent space” ${\mathbb{R}}^{\mathrm{\ell }}$, an autoencoder is a pair of continuous maps consisting of an “encoder” $E\text{\hspace{0.17em}⁣}:{\mathbb{R}}^n\to {\mathbb{R}}^{\mathrm{\ell }}$ and “decoder” $D\text{\hspace{0.17em}⁣}:{\mathbb{R}}^{\mathrm{\ell }}\to {\mathbb{R}}^n$ such that the “round trip” map $D\circ E$ is as close as possible to the identity map $i{d}_M$ on $M$. We present various topological limitations and capabilites inherent to the search for an autoencoder, and describe capabilities for autoencoding dynamical systems having $M$ as an invariant manifold.