Information-Computation Tradeoffs for Noiseless Linear Regression with Oblivious Contamination

We study the task of noiseless linear regression under Gaussian covariates in the presence of additive oblivious contamination. Specifically, we are given i.i.d.\ samples from a distribution $(x,y)$ on ${\mathbb{R}}^d\times \mathbb{R}$ with $x\sim \mathcal{N}(0,{\mathbf{I}}_d)$ and $y=x^{\mathrm{\top }}\beta +z$, where $z$ is drawn independently of $x$ from an unknown distribution $E$. Moreover, $z$ satisfies ${\mathbb{P}}_E[z=0]=\alpha >0$. The goal is to accurately recover the regressor $\beta$ to small ${\mathrm{\ell }}_2$-error. Ignoring computational considerations, this problem is known to be solvable using $O(d\mathrm{/}\alpha )$ samples. On the other hand, the best known polynomial-time algorithms require $\mathrm{\Omega }(d\mathrm{/}{\alpha }^2)$ samples. Here we provide formal evidence that the quadratic dependence in $1\mathrm{/}\alpha$ is inherent for efficient algorithms. Specifically, we show that any efficient Statistical Query algorithm for this task requires VSTAT complexity at least $\tilde{\mathrm{\Omega }}(d^{1\mathrm{/}2}\mathrm{/}{\alpha }^2)$.