Mini-Minimax Uncertainty Quantification for Emulators

Consider approximating a “black box” function $f$ by an emulator $\widehat{f}$ based on $n$ noiseless observations of $f$. Let $w$ be a point in the domain of $f$. How big might the error $\mathrm{\mid }\widehat{f}(w)-f(w)\mathrm{\mid }$ be? If $f$ could be arbitrarily rough, this error could be arbitrarily large: we need some constraint on $f$ besides the data. Suppose $f$ is Lipschitz with known constant. We find a lower bound on the number of observations required to ensure that for the best emulator $\widehat{f}$ based on the $n$ data, $|\hat{f}(w)

• f(w)| \le \epsilon$\mathrm{.}Butingeneral,wewillnotknowwhether$f$ is Lipschitz, much less know its Lipschitz constant. Assume optimistically that $f$ is Lipschitz-continuous with the smallest constant consistent with the $n$ data. We find the maximum (over such regular $f$) of $\mathrm{\mid }\widehat{f}(w)-f(w)\mathrm{\mid }$ for the best possible emulator $\widehat{f}$; we call this the “mini-minimax uncertainty” at $w$. In reality, $f$ might not be Lipschitz or—if it is—it might not attain its Lipschitz constant on the data. Hence, the mini-minimax uncertainty at $w$ could be much smaller than $\mathrm{\mid }\widehat{f}(w)-f(w)\mathrm{\mid }$. But if the mini-minimax uncertainty is large, then—even if $f$ satisfies the optimistic regularity assumption—$\mathrm{\mid }\widehat{f}(w)-f(w)\mathrm{\mid }$ could be large, no matter how cleverly we choose $\widehat{f}$. For the Community Atmosphere Model, the maximum (over $w$) of the mini-minimax uncertainty based on a set of 1154~observations of $f$ is no smaller than it would be for a single observation of $f$ at the centroid of the 21-dimensional parameter space. We also find lower confidence bounds for quantiles of the mini-minimax uncertainty and its mean over the domain of $f$. For the Community Atmosphere Model, these lower confidence bounds are an appreciable fraction of the maximum.