Uncertainty Quantification UQ
Definition
The discipline of characterizing the sources and magnitude of uncertainty in model predictions: separating input-parameter uncertainty (aleatoric) from model uncertainty (epistemic), and producing decision-usable confidence intervals rather than a single point prediction.
What it is
UQ answers three questions: where uncertainty comes from (input parameters, model form, numerical error, data noise), how large it is (variance, quantiles, confidence intervals), and how it propagates (how input uncertainty amplifies into output uncertainty). For AI surrogates, UQ is the gatekeeper for production use.
Key points
- Two families:
aleatoric(inherent randomness in data or parameters, not reduced by more data) andepistemic(model ignorance, reduced by more data or a better model). They call for different treatments. - Common methods: Gaussian processes give variance for free, deep ensembles, Monte Carlo Dropout, and conformal prediction (which carries coverage guarantees).
- Calibration matters as much: if you claim a 90% interval, verify with reliability diagrams that empirical coverage is really 90%.
- It pairs naturally with active learning: sample where uncertainty is high and you improve the model with the fewest samples.
Common pitfalls / boundaries
- A point prediction plus average error is not decision-usable; you must provide intervals or quantiles.
- A neural network's "softmax probability" is not uncertainty; deep models need dedicated UQ machinery.