Sparse single-index model

Let $(\text{bX},Y)$ be a random pair taking values in $\mathbb R^p \times \mathbb R$\mathrm{.}Intheso-calledsingle-indexmodel,onehas$Y=f^{\star}(\theta^{\star T}\bX)+\bW$,where$f^{\star}$isanunknownunivariatemeasurablefunction,$\theta^{\star}$isanunknownvectorin$\mathbb R^d$,and$W$ denotes a random noise satisfying $\mathbb{E}[\text{bW}\mathrm{\mid }\text{bX}]=0$. The single-index model is known to offer a flexible way to model a variety of high-dimensional real-world phenomena. However, despite its relative simplicity, this dimension reduction scheme is faced with severe complications as soon as the underlying dimension becomes larger than the number of observations (“$p$ larger than $n$” paradigm). To circumvent this difficulty, we consider the single-index model estimation problem from a sparsity perspective using a PAC-Bayesian approach. On the theoretical side, we offer a sharp oracle inequality, which is more powerful than the best known oracle inequalities for other common procedures of single-index recovery. The proposed method is implemented by means of the reversible jump Markov chain Monte Carlo technique and its performance is compared with that of standard procedures.