GIT-BO

Explore

GIT-BO (ICLR 2026) turns a pre-trained tabular foundation model into a retraining-free surrogate for high-dimensional Bayesian optimization, ranking first against state-of-the-art Gaussian-process methods on problems with up to 500 dimensions.

Paper

Code

arXiv

Bayesian Optimization Without Retraining

Bayesian optimization struggles in high dimensions, where Gaussian-process surrogates demand heavy retraining and brittle structural assumptions. GIT-BO replaces the GP entirely with TabPFN v2, a tabular foundation model that performs zero-shot Bayesian inference in context, so the surrogate never needs online retraining during the optimization loop.

A Subspace Found for Free

GIT-BO exploits gradients the foundation model already computes in its forward pass. A gradient-informed diagnostic matrix, built from the model's own predictive-mean gradients via a Fisher-information estimate, reveals the most sensitive directions of the search space and continuously re-estimates a low-dimensional active subspace — no fixed embedding, no structural assumptions.

Ranked First Across 60 Problem Variants

Across 60 problem variants spanning 20 benchmarks — nine scalable synthetic families and eleven real-world engineering tasks including power systems, Rover, MOPTA08, and Mazda vehicle design — GIT-BO consistently outperforms SAASBO, TuRBO, Vanilla BO, and BAxUS, and its advantage grows with dimensionality up to 500D. It ranks first overall on real-world engineering problems.

Better Optima in Minutes, Not Hours

Because the surrogate is never retrained, GIT-BO converges in minutes where GP-based high-dimensional BO methods take hours, delivering a Pareto-dominant performance-versus-time trade-off whose runtime advantage grows with dimensionality.

Citations