Efficiency requires innovation

In estimation a parameter $\theta \in \mathbb{R}$ from a sample $(x_1,\dots ,x_n)$ from a population $P_{\theta }$ a simple way of incorporating a new observation $x_{n+1}$ into an estimator $\tilde\theta_{n} = \tilde\theta_{n}(x_1,\ldots,x_n)$istransforming$\tilde\theta_n$ to what we call the {\it jackknife extension} $\tilde\theta_{n+1}^{(e)} = \tilde\theta_{n+1}^{(e)}(x_1,\ldots,x_n,x_{n+1})$, $${\tilde{\theta }}_{n+1}^{(e)}=\{{\tilde{\theta }}_n(x_1,\dots ,x_n)+{\tilde{\theta }}_n(x_{n+1},x_2,\dots ,x_n)+\dots +{\tilde{\theta }}_n(x_1,\dots ,x_{n-1},x_{n+1})\}\mathrm{/}(n+1)\mathrm{.}$$ Though ${\tilde{\theta }}_{n+1}^{(e)}$ lacks an innovation the statistician could expect from a larger data set, it is still better than ${\tilde{\theta }}_n$, $$\mathrm{v}\mathrm{a}\mathrm{r}({\tilde{\theta }}_{n+1}^{(e)})\le \frac{n}{n+1}\mathrm{v}\mathrm{a}\mathrm{r}({\tilde{\theta }}_n)\mathrm{.}$$ However, an estimator obtained by jackknife extension for all $n$ is asymptotically efficient only for samples from exponential families. For a general $P_{\theta }$, asymptotically efficient estimators require innovation when a new observation is added to the data. Some examples illustrate the concept.