FIRE
FIRE: Multi-Fidelity Regression with Distribution-Conditioned In-Context Learning Using Tabular Foundation Models
1Center for Computational Science and Engineering, MIT 2Department of Mechanical Engineering, MIT
FIRE: Multi-Fidelity Regression with Distribution-Conditioned In-Context Learning Using Tabular Foundation Models
1Center for Computational Science and Engineering, MIT 2Department of Mechanical Engineering, MIT
FIRE (ICML 2026, Spotlight) is a training-free multi-fidelity regression framework that couples tabular foundation models through distribution-conditioned in-context learning, beating seven state-of-the-art baselines in accuracy, uncertainty quantification, and runtime simultaneously.
Multi-fidelity regression usually operates under extreme data imbalance, where a handful of expensive high-fidelity samples must be fused with abundant cheap low-fidelity data. Gaussian-process surrogates suffer cubic scaling costs and overfit the sparse high-fidelity set. FIRE instead performs zero-shot in-context Bayesian inference with a tabular foundation model, conditioning a high-fidelity correction model on the low-fidelity model's full posterior predictive distribution — mean, variance, and quantiles — so cross-fidelity information transfers through distributional summaries without any model training.
Across 31 benchmarks spanning synthetic functions, engineering simulations (DrivAerNet, Blended Wing Body), and hyperparameter-optimization tasks (LCBench), FIRE beats seven state-of-the-art GP-based and deep-learning methods in accuracy and uncertainty quantification and runtime, achieving an average rank of 1.8 versus 3.5 for the strongest baseline. FIRE is both the fastest and the most accurate, dominating the Pareto frontier for accuracy and uncertainty alike.
FIRE's margin is largest in the most punishing regime — when high-fidelity samples make up only 2 to 5 percent of the data, exactly where heteroscedastic errors break residual-learning baselines and where real engineering budgets live. Removing the distributional conditioning (using mean-only residuals, as GP methods do) costs roughly 250 Elo points.
FIRE was evaluated on seven physics-based engineering datasets. Pick one to see its results from the paper — NRMSE, NLL, and R² against all baselines as the high-fidelity data budget grows from 2% to 25% of the low-fidelity data (FIRE in red).
Drag-coefficient prediction for a blended-wing-body aircraft from multi-fidelity aerodynamic simulations. A new multi-fidelity dataset introduced with FIRE.
Lift-coefficient prediction for a blended-wing-body aircraft from multi-fidelity aerodynamic simulations. A new multi-fidelity dataset introduced with FIRE.
Drag regression pairing high-fidelity automotive CFD from DrivAerNet with a low-fidelity model. A new multi-fidelity dataset introduced with FIRE.
Multi-fidelity prediction of concrete compressive strength from mixture proportions. A new multi-fidelity dataset introduced with FIRE.
Parametric aircraft wing-weight model, a physics-based parametric multi-fidelity benchmark.
Structural beam deflection, a physics-based parametric multi-fidelity benchmark.
Hybrid organic-inorganic perovskite band-gap prediction, a three-fidelity physics-based benchmark.
Yu, Rosen Ting-Ying, Nicholas Sung, and Faez Ahmed. 'FIRE: Multi-Fidelity Regression with Distribution-Conditioned In-Context Learning Using Tabular Foundation Models.' In International Conference on Machine Learning (ICML). 2026.
@inproceedings{yu2026fire,
title={FIRE: Multi-Fidelity Regression with Distribution-Conditioned In-Context Learning Using Tabular Foundation Models},
author={Yu,
Rosen Ting-Ying and Sung,
Nicholas and Ahmed,
Faez},
booktitle={International Conference on Machine Learning},
year={2026}}
