Robust Bayesian Optimization with Q-Exponential Surrogates

q-ED-BO separates two jobs that are coupled in ordinary GP-BO. The kernel posterior still says where the objective is likely to be high and where data are sparse. The q-exponential predictive says how strongly the optimizer should trust that local picture when evaluations can be corrupted. This separation preserves the computational structure of GP-BO while changing the uncertainty that drives the next query.

Bayesian optimization searches for the optimum of an expensive black box with a probabilistic surrogate and an acquisition function¹. Gaussian processes are attractive because their posterior location and scale are available in closed form². The Gaussian assumption is also brittle. A single corrupted evaluation can move the posterior mean and inflate uncertainty around a spurious region³. Because the acquisition reads that predictive distribution directly, the optimizer can spend later evaluations chasing the corruption.

• Heavy-tailed likelihood. A Student-\(t\) likelihood can reduce the influence of an outlying observation, but the GP and Student-\(t\) pair is not conjugate, so inference is approximate⁴.
• Heavy-tailed prior. A Student-\(t\) process gives an exact heavy-tailed predictive, but its polynomial tails do not provide the exponential concentration used by UCB regret analysis⁵.
• Truncated rewards. Truncation controls heavy-tailed observation noise, but it does not directly model a corrupted objective predictive⁶.

The design target is therefore specific. The predictive itself should be heavy-tailed, analytically tractable, and concentrated enough to calibrate a confidence bound. The q-exponential family supplies all three properties.

Let \(\mu_t(\mathbf{x})\) and \(\sigma_t^2(\mathbf{x})\) be the usual GP posterior quantities after \(t\) evaluations. q-ED-BO uses them as the location and scale of a univariate q-exponential distribution⁷.

The distinction between scale and variance matters. The predictive mean is \(\mu_t(\mathbf{x})\), while its variance is \(c(q)\sigma_t^2(\mathbf{x})\), where

At \(q=2\), \(c(2)=1\) and the predictive is exactly Gaussian. For \(0\lt q\lt 2\), the density is more peaked at its mode and decays more slowly in the tails. Decreasing \(q\) therefore changes both the shape of uncertainty and its variance. It does not move the GP location, change the kernel, or require a new matrix inverse. This is stronger than merely inflating a Gaussian variance because variance inflation leaves the Gaussian tail shape unchanged.

A useful representation makes the construction intuitive. If \(U\sim q\text{-ED}_1(m,\sigma^2)\), then \(U=m+R\sigma S\), where \(S\) is equally likely to be \(-1\) or \(+1\) and \(R^q\) follows a chi-squared distribution with one degree of freedom. Equivalently, \(R\) has the same distribution as \(|N|^{2/q}\) for a standard normal \(N\). The parameter \(q\) acts only through this radial transformation.

The derivation is short because of the radial representation. The event \(|U-m|>r\sigma\) is the same as \(R>r\). Substituting \(R=|N|^{2/q}\) turns it into the Gaussian event \(|N|>r^{q/2}\). Applying the standard Gaussian tail bound gives the stretched-exponential decay above. No numerical integration is needed.

To make this guarantee hold for every candidate point and every round, the paper assigns failure probability \(\delta_t=6\delta/(\pi^2t^2)\) and applies a union bound. Inverting the lemma then gives

This is the key bridge from distributional robustness to an acquisition rule. Smaller \(q\) means a larger width \(\rho_t\), so the optimizer explores more aggressively when the predictive assigns greater probability to extreme deviations.

q-UCB inserts the new confidence width into the familiar GP-UCB score⁸. At round \(t\), it selects the point maximizing

Here \(\gamma_T\) is the kernel's maximum information gain and \(C_1\) is the usual information-gain constant. The proof follows the standard UCB route. The tail lemma creates a simultaneous confidence event, instantaneous regret is at most \(2\rho_t\sigma_{t-1}(\mathbf{x}_t)\), information gain controls the sum of posterior variances, and Cauchy-Schwarz converts those per-round bounds into the cumulative result.

At \(q=2\), \(\rho_t=\Lambda_t^{1/2}\) and the classical GP-UCB rate is recovered. For \(q\lt 2\), the confidence width grows by \(\Lambda_t^{1/q-1/2}\). This is the precise price of a heavier predictive tail. The penalty is logarithmic, while the leading \(\sqrt{T\gamma_T}\) dependence is unchanged.

q-EI scores the expected positive improvement over the best observed value \(f^+\)⁹. Define the standardized gap \(d=(\mu_t(\mathbf{x})-f^+)/\sigma_t(\mathbf{x})\). Under the q-exponential predictive, the standardized CDF and upper partial first moment are

Standardizing the improvement gives \(\mathrm{E}[(d+Z)_+]\). Symmetry splits this expectation into a location term, \(dF_q(d)\), and an upper-tail partial moment. The substitution \(v=z^q/2\) evaluates that remaining integral as \(M_q(|d|)\). The resulting acquisition is exact.

When \(q=2\), \(F_2\) becomes the Gaussian CDF and \(M_2\) becomes the Gaussian density, exactly recovering classical EI. Unlike q-UCB, q-EI does not yet have a regret guarantee. Even Gaussian EI does not admit the same direct confidence-interval argument, so extending existing consistency results remains open¹⁰.

1. Evaluate a small initial design and form the first dataset.
2. Refit kernel hyperparameters with \(q\) fixed, then compute the GP posterior location and scale^a.
3. Choose the next query by maximizing q-UCB or q-EI.
4. Evaluate the black box and append the new observation.
5. Repeat until the budget is exhausted, then return the best observed point.

Only the predictive tail and the acquisition score change. The method remains compatible with any valid kernel, including kernels adapted during the search¹¹. The experiments suggest \(q=1.5\) as a safe default. Smaller values create heavier tails and stronger exploration, which can help under severe corruption but can over-explore when the data are clean.

Both tasks treat a full signal-processing pipeline as the black box. The beamformer returns output SINR. The adaptive filter returns steady-state misalignment reduction. Higher is better in both cases. Every method uses 5 Sobol starts, a budget of 100 queries, a squared-exponential kernel, L-BFGS to maximize the acquisition, and 10 seeds. Baselines are GP-UCB, GP-EI, Student-\(t\)-process EI, and truncated-mean GP-UCB, each compared with \(q\)-UCB and \(q\)-EI at \(q\in\{1.5,1,0.5\}\).

The objective is the output SINR of a robust minimum-variance distortionless-response beamformer¹², over four design parameters: diagonal loading, look direction, the robust Capon uncertainty radius, and covariance shrinkage. The array is a 16-element uniform linear array with 24 snapshots, a signal of interest at broadside (SNR 10 dB), and four interferers (INR 25 dB), one of them near the mainlobe. Clean snapshots are Gaussian. In the impulsive regime, burst and Cauchy outliers (peak rate 0.10 near the interferer directions, floor 0.05) corrupt both the sample covariance and the online SINR meter, together with meter noise of standard deviation 0.35 dB.

On clean data the methods largely agree. GP-UCB, Student-\(t\)-process EI, truncated-mean GP-UCB, and \(q\)-UCB at \(q=1.5\) finish within about 0.2 dB of each other, and \(q\)-EI at \(q=1\) is best overall, so a mildly heavy-tailed predictive is essentially free when there are no outliers. At \(q=0.5\), \(q\)-UCB loses about 1 dB to GP-UCB because the wider intervals explore too much. Under impulsive corruption, \(q\)-UCB and \(q\)-EI at \(q=1.5\) beat truncated-mean GP-UCB, the strongest baseline, by about 0.7 dB. Those gains are significant under a paired Wilcoxon signed-rank test across seeds (\(p\lt0.05\))¹³. Truncation robustifies the fit and leaves a Gaussian predictive on the objective, so the search can still lock onto a corrupted region. The setting \(q=0.5\) again over-explores.

The second task tunes a leaky normalized least-mean-squares filter for system identification¹⁴. A length-16 plant is driven by white Gaussian input. The desired signal is observed in Gaussian noise, or in ε-contaminated Gaussian and Cauchy noise with contamination rate 0.10. A single spike enters the tap update directly, which is the usual failure mode of the LMS family. The black box has three parameters, the step size, the regularization, and the leakage, and it returns the steady-state misalignment reduction in dB from a single run, corrupted by meter noise and occasional Cauchy spikes.

On clean data every method except \(q\)-UCB at \(q=0.5\) finishes within about 0.5 dB of the others and within about 1 dB of the 31 dB grid optimum. The \(q\)-UCB method at \(q=1.5\) is slightly ahead, and \(q\)-EI at \(q=1.5\) matches GP-EI to the reported precision. Under impulsive noise every method loses roughly half of the attainable reduction. The \(q\)-UCB and \(q\)-EI methods at \(q=1.5\) still recover 1.1 to 1.2 dB more than truncated-mean GP-UCB (Table 1, right), and the same Wilcoxon test finds those \(q=1.5\) gains significant. The \(q\)-EI method at \(q=1.5\) finishes within 0.2 dB of \(q\)-UCB. The paper recommends \(q=1.5\) as a default, and a smaller \(q\) only when corruption is severe.

q-ED-BO puts the heavy tail in the predictive rather than only in the likelihood, and \(q\)-UCB keeps a sublinear regret bound, with only a logarithmic extra cost as \(q\) decreases. The shape \(q\) is fixed before the search, and \(q\)-EI is available in closed form but has no regret guarantee, so learning \(q\) from the data, and a convergence rate for \(q\)-EI, remain open.