English
Related papers

Related papers: Certifiable Deep Importance Sampling for Rare-Even…

200 papers

We discuss estimating the probability that the sum of nonnegative independent and identically distributed random variables falls below a given threshold, i.e., $\mathbb{P}(\sum_{i=1}^{N}{X_i} \leq \gamma)$, via importance sampling (IS). We…

Computation · Statistics 2021-10-04 Nadhir Ben Rached , Abdul-Lateef Haji-Ali , Gerardo Rubino , Raul Tempone

We propose a class of strongly efficient rare event simulation estimators for random walks and compound Poisson processes with a regularly varying increment/jump-size distribution in a general large deviations regime. Our estimator is based…

Probability · Mathematics 2017-06-14 Bohan Chen , Jose Blanchet , Chang-Han Rhee , Bert Zwart

Extreme weather events epitomize high cost: to society through their physical impacts, and to computer servers that simulate them to assess risk and advance physical understanding. It costs hundreds of simulation years to sample a few…

Atmospheric and Oceanic Physics · Physics 2026-04-14 Justin Finkel , Paul A. O'Gorman

Algorithms for rare event complex systems simulations are proposed. Compressed Sensing (CS) has {\it revolutionized} our understanding of limits in signal recovery and has forced us to re-define Shannon-Nyquist sampling theorem for sparse…

Computational Physics · Physics 2018-04-27 Mehmet Süzen

Many high-stakes AI applications target low-prevalence events, where apparent accuracy can conceal limited real-world value. Relevant AI models range from expert-defined rules and traditional machine learning to generative LLMs constrained…

Machine Learning · Computer Science 2025-10-07 G. Niklas Noren , Eva-Lisa Meldau , Johan Ellenius

Driven by applications in telecommunication networks, we explore the simulation task of estimating rare event probabilities for tandem queues in their steady state. Existing literature has recognized that importance sampling methods can be…

Machine Learning · Computer Science 2025-04-22 Ruoning Zhao , Xinyun Chen

Evaluating rare but high-stakes events is one of the main challenges in obtaining reliable reinforcement learning policies, especially in large or infinite state/action spaces where limited scalability dictates a prohibitively large number…

Machine Learning · Computer Science 2022-10-04 Mengdi Xu , Peide Huang , Fengpei Li , Jiacheng Zhu , Xuewei Qi , Kentaro Oguchi , Zhiyuan Huang , Henry Lam , Ding Zhao

Article describes the results of the development and using of Rare-Event Monte-Carlo Simulation Algorithms for Dynamic Fault Trees Estimation. For Fault Trees estimation usually analytical methods are used (Minimal Cut sets, Markov Chains,…

Applications · Statistics 2016-01-28 Sergey Porotsky

Many contemporary machine learning models require extensive tuning of hyperparameters to perform well. A variety of methods, such as Bayesian optimization, have been developed to automate and expedite this process. However, tuning remains…

Machine Learning · Computer Science 2020-02-25 Setareh Ariafar , Zelda Mariet , Ehsan Elhamifar , Dana Brooks , Jennifer Dy , Jasper Snoek

Importance sampling (IS) as an elegant and efficient variance reduction (VR) technique for the acceleration of stochastic optimization problems has attracted many researches recently. Unlike commonly adopted stochastic uniform sampling in…

Machine Learning · Computer Science 2017-11-02 Fei Wang , Xiaofeng Gao , Guihai Chen , Jun Ye

Deep neural networks (DNNs) have demonstrated superior performance over classical machine learning to support many features in safety-critical systems. Although DNNs are now widely used in such systems (e.g., self driving cars), there is…

Software Engineering · Computer Science 2022-10-17 Mohammed Oualid Attaoui , Hazem Fahmy , Fabrizio Pastore , Lionel Briand

Detecting rare events, those defined to give rise to high impact but have a low probability of occurring, is a challenge in a number of domains including meteorological, environmental, financial and economic. The use of machine learning to…

Applications · Statistics 2022-09-13 Santhosh Narayanan , Carsten Maple , Mark Hooper

In this paper, we introduce a new algorithm for rare event estimation based on adaptive importance sampling. We consider a smoothed version of the optimal importance sampling density, which is approximated by an ensemble of interacting…

Computation · Statistics 2023-04-19 Konstantin Althaus , Iason Papaioannou , Elisabeth Ullmann

Dynamic Fault Trees (DFT) are widely adopted in industry to assess the dependability of safety-critical equipment. Since many systems are too large to be studied numerically, DFTs dependability is often analysed using Monte Carlo…

Distributed, Parallel, and Cluster Computing · Computer Science 2019-10-29 Carlos E. Budde , Marco Biagi , Raúl E. Monti , Pedro R. D'Argenio , Mariëlle Stoelinga

We explore efficient estimation of statistical quantities, particularly rare event probabilities, for stochastic reaction networks. Consequently, we propose an importance sampling (IS) approach to improve the Monte Carlo (MC) estimator…

Numerical Analysis · Mathematics 2024-03-12 Chiheb Ben Hammouda , Nadhir Ben Rached , Raúl Tempone , Sophia Wiechert

Rare events, despite their infrequency, often carry critical information and require immediate attentions in mission-critical applications such as autonomous driving, healthcare, and industrial automation. The data-intensive nature of these…

Machine Learning · Computer Science 2025-01-07 You Zhou , Changsheng You , Kaibin Huang

A particularly challenging problem in AI safety is providing guarantees on the behavior of high-dimensional autonomous systems. Verification approaches centered around reachability analysis fail to scale, and purely statistical approaches…

Artificial Intelligence · Computer Science 2025-03-11 Souradeep Dutta , Michele Caprio , Vivian Lin , Matthew Cleaveland , Kuk Jin Jang , Ivan Ruchkin , Oleg Sokolsky , Insup Lee

The development of enhanced sampling methods has greatly extended the scope of atomistic simulations, allowing long-time phenomena to be studied with accessible computational resources. Many such methods rely on the identification of an…

Computational Physics · Physics 2022-06-08 Luigi Bonati , GiovanniMaria Piccini , Michele Parrinello

We consider the problem of efficient simulation estimation of the density function at the tails, and the probability of large deviations for a sum of independent, identically distributed, light-tailed and non-lattice random vectors. The…

Probability · Mathematics 2017-04-26 Santanu Dey , Sandeep Juneja , Ankush Agarwal

We consider the problem of choosing design parameters to minimize the probability of an undesired rare event that is described through the average of $n$ iid random variables. Since the probability of interest for near optimal design…

Optimization and Control · Mathematics 2019-02-22 Amarjit Budhiraja , Shu Lu , Yang Yu , Quoc Tran-Dinh