English
Related papers

Related papers: Simulation-based Calibration of Uncertainty Interv…

200 papers

Calibration error is commonly adopted for evaluating the quality of uncertainty estimators in deep neural networks. In this paper, we argue that such a metric is highly beneficial for training predictive models, even when we do not…

Machine Learning · Statistics 2019-11-01 Jayaraman J. Thiagarajan , Bindya Venkatesh , Deepta Rajan

Bayesian inference allows expressing the uncertainty of posterior belief under a probabilistic model given prior information and the likelihood of the evidence. Predominantly, the likelihood function is only implicitly established by a…

Understanding the uncertainty of a neural network's (NN) predictions is essential for many purposes. The Bayesian framework provides a principled approach to this, however applying it to NNs is challenging due to large numbers of parameters…

Machine Learning · Statistics 2020-02-27 Tim Pearce , Felix Leibfried , Alexandra Brintrup , Mohamed Zaki , Andy Neely

Modern neural networks have proven to be powerful function approximators, providing state-of-the-art performance in a multitude of applications. They however fall short in their ability to quantify confidence in their predictions - this is…

Machine Learning · Statistics 2020-06-29 Alex J. Chan , Ahmed M. Alaa , Zhaozhi Qian , Mihaela van der Schaar

Verifying the correctness of Bayesian computation is challenging. This is especially true for complex models that are common in practice, as these require sophisticated model implementations and algorithms. In this paper we introduce…

Methodology · Statistics 2020-10-22 Sean Talts , Michael Betancourt , Daniel Simpson , Aki Vehtari , Andrew Gelman

Nested error regression models are useful tools for analysis of grouped data, especially in the case of small area estimation. This paper suggests a nested error regression model using uncertain random effects in which the random effect in…

Methodology · Statistics 2017-02-28 Shonosuke Sugasawa , Tatsuya Kubokawa

Bayesian Neural Networks (BNNs) offer a principled and natural framework for proper uncertainty quantification in the context of deep learning. They address the typical challenges associated with conventional deep learning methods, such as…

Computation · Statistics 2024-11-13 Zahra Moslemi , Yang Meng , Shiwei Lan , Babak Shahbaba

We develop a Bayesian approach called Bayesian projected calibration to address the problem of calibrating an imperfect computer model using observational data from a complex physical system. The calibration parameter and the physical…

Methodology · Statistics 2019-02-08 Fangzheng Xie , Yanxun Xu

Probabilistic prediction of sequences from images and other high-dimensional data is a key challenge, particularly in risk-sensitive applications. In these settings, it is often desirable to quantify the uncertainty associated with the…

Machine Learning · Computer Science 2024-10-31 Qidong Yang , Weicheng Zhu , Joseph Keslin , Laure Zanna , Tim G. J. Rudner , Carlos Fernandez-Granda

We present a multi-fidelity method for uncertainty quantification of parameter estimates in complex systems, leveraging generative models trained to sample the target conditional distribution. In the Bayesian inference setting, traditional…

Machine Learning · Computer Science 2025-04-03 Caroline Tatsuoka , Minglei Yang , Dongbin Xiu , Guannan Zhang

Computer model calibration is a crucial step in building a reliable computer model. In the face of massive physical observations, a fast estimation for the calibration parameters is urgently needed. To alleviate the computational burden, we…

Statistics Theory · Mathematics 2022-11-24 Shurui Lv , Yan Wang , Jun Yu

A common method for assessing validity of Bayesian sampling or approximate inference methods makes use of simulated data replicates for parameters drawn from the prior. Under continuity assumptions, quantiles of functions of the simulated…

Computation · Statistics 2019-11-21 Xuejun Yu , David J. Nott , Minh-Ngoc Tran , Nadja Klein

The steady-state Bayesian vector autoregression (BVAR) makes it possible to incorporate prior information about the long-run mean of the process. This has been shown in many studies to substantially improve forecasting performance, and the…

Computation · Statistics 2025-06-12 Oskar Gustafsson , Mattias Villani

Ensemble forecasts of weather and climate are subject to systematic biases in the ensemble mean and variance, leading to inaccurate estimates of the forecast mean and variance. To address these biases, ensemble forecasts are post-processed…

Applications · Statistics 2016-05-25 Stefan Siegert , Philip G. Sansom , Robin Williams

Accurate uncertainty estimates can significantly improve the performance of iterative design of experiments, as in Sequential and Reinforcement learning. For many such problems in engineering and the physical sciences, the design task…

Machine Learning · Statistics 2022-05-20 Brendan Folie , Maxwell Hutchinson

A key factor in ensuring the accuracy of computer simulations that model physical systems is the proper calibration of their parameters based on real-world observations or experimental data. Inevitably, uncertainties arise, and Bayesian…

Computational Engineering, Finance, and Science · Computer Science 2026-02-25 Daniel Andrés Arcones , Martin Weiser , Phaedon-Stelios Koutsourelakis , Jörg F. Unger

A vital stage in the mathematical modelling of real-world systems is to calibrate a model's parameters to observed data. Likelihood-free parameter inference methods, such as Approximate Bayesian Computation, build Monte Carlo samples of the…

Computation · Statistics 2021-12-23 Thomas P Prescott , Ruth E Baker

In this article we present very intuitive, easy to follow, yet mathematically rigorous, approach to the so called data fitting process. Rather than minimizing the distance between measured and simulated data points, we prefer to find such…

Data Analysis, Statistics and Probability · Physics 2017-08-07 Marek W. Gutowski

Time series analysis of fMRI data is an important area of medical statistics for neuroimaging data. The neuroimaging community has embraced mean-field variational Bayes (VB) approximations, which are implemented in Statistical Parametric…

Computation · Statistics 2018-03-06 Ming Teng , Timothy Johnson , Farouk Nathoo

We study a mean-field spike and slab variational Bayes (VB) approximation to Bayesian model selection priors in sparse high-dimensional linear regression. Under compatibility conditions on the design matrix, oracle inequalities are derived…

Methodology · Statistics 2020-11-20 Kolyan Ray , Botond Szabo