English
Related papers

Related papers: Sampling rare events: statistics of local sequence…

200 papers

While score-based generative models (SGMs) have achieved remarkable success in enormous image generation tasks, their mathematical foundations are still limited. In this paper, we analyze the approximation and generalization of SGMs in…

Machine Learning · Statistics 2024-02-26 Frank Cole , Yulong Lu

We consider the Random Euclidean Assignment Problem in dimension $d=1$, with linear cost function. In this version of the problem, in general, there is a large degeneracy of the ground state, i.e. there are many different optimal matchings…

Probability · Mathematics 2021-07-16 Sergio Caracciolo , Vittorio Erba , Andrea Sportiello

Specifying a proper input distribution is often a challenging task in simulation modeling. In practice, there may be multiple plausible distributions that can fit the input data reasonably well, especially when the data volume is not large.…

Methodology · Statistics 2019-03-15 Weiwei Fan , L. Jeff Hong , Xiaowei Zhang

The Adaptive Multilevel Splitting algorithm is a very powerful and versatile method to estimate rare events probabilities. It is an iterative procedure on an interacting particle system, where at each step, the $k$ less well-adapted…

Probability · Mathematics 2014-05-07 Charles-Edouard Bréhier , Tony Lelievre , Mathias Rousset

Recent likelihood theory produces $p$-values that have remarkable accuracy and wide applicability. The calculations use familiar tools such as maximum likelihood values (MLEs), observed information and parameter rescaling. The usual…

Methodology · Statistics 2008-02-08 M. Bédard , D. A. S. Fraser , A. Wong

This undergraduate thesis focuses on calculating maximum likelihood estimates of parameters in the generalized Gamma distribution using the SeLF algorithm. As an extension of the Gamma distribution, the generalized Gamma distribution can…

Methodology · Statistics 2026-01-01 Yufei Cai

When learning from positive and unlabelled data, it is a strong assumption that the positive observations are randomly sampled from the distribution of $X$ conditional on $Y = 1$, where X stands for the feature and Y the label. Most…

Machine Learning · Computer Science 2020-03-04 Fengxiang He , Tongliang Liu , Geoffrey I Webb , Dacheng Tao

An empirical Bayes approach to the estimation of possibly sparse sequences observed in Gaussian white noise is set out and investigated. The prior considered is a mixture of an atom of probability at zero and a heavy-tailed density \gamma,…

Statistics Theory · Mathematics 2007-06-13 Iain M. Johnstone , Bernard W. Silverman

Modern data sets in various domains often include units that were sampled non-randomly from the population and have a latent correlation structure. Here we investigate a common form of this setting, where every unit is associated with a…

Methodology · Statistics 2019-07-25 Omer Weissbrod , Shachar Kaufman , David Golan , Saharon Rosset

We consider two independent binary i.i.d. random strings $X$ and $Y$ of equal length $n$ and the optimal alignments according to a symmetric scoring functions only. We decompose the space of scoring functions into five components. Two of…

Probability · Mathematics 2017-08-02 Jun Tao Duan , Heinrich Matzinger , Ionel Popescu

Using statistical thermodynamics, we derive a general expression of the stationary probability distribution for thermodynamic systems driven out of equilibrium by several thermodynamic forces. The local equilibrium is defined by imposing…

Statistical Mechanics · Physics 2015-06-12 Giorgio Sonnino , György Steinbrecher , Alessandro Cardinali , Alberto Sonnino , Mustapha Tlidi

This paper introduces a new Importance Sampling scheme, called Adaptive Twisted Importance Sampling, which is adequate for the improved estimation of rare event probabilities in he range of moderate deviations pertaining to the empirical…

Computation · Statistics 2009-10-13 Michel Broniatowski , Ya'Acov Ritov

Multilevel Splitting methods, also called Sequential Monte-Carlo or \emph{Subset Simulation}, are widely used methods for estimating extreme probabilities of the form $P[S(\mathbf{U}) > q]$ where $S$ is a deterministic real-valued function…

Computation · Statistics 2015-07-06 Clément Walter

This paper provides a detailed introductory description of Subset Simulation, an advanced stochastic simulation method for estimation of small probabilities of rare failure events. A simple and intuitive derivation of the method is given…

Computation · Statistics 2015-05-14 Konstantin Zuev

Suppose an interval is put on a horizontal line with random roughness. With probability one it is supported at two points, one from the left, and another from the right from its center. We compute probability distribution of support points…

Probability · Mathematics 2013-05-20 Dmitry Treschev

Low-dimensional embedding, manifold learning, clustering, classification, and anomaly detection are among the most important problems in machine learning. The existing methods usually consider the case when each instance has a fixed,…

Machine Learning · Computer Science 2012-02-20 Barnabas Poczos , Liang Xiong , Jeff Schneider

In image segmentation, there is often more than one plausible solution for a given input. In medical imaging, for example, experts will often disagree about the exact location of object boundaries. Estimating this inherent uncertainty and…

Computer Vision and Pattern Recognition · Computer Science 2020-12-23 Miguel Monteiro , Loïc Le Folgoc , Daniel Coelho de Castro , Nick Pawlowski , Bernardo Marques , Konstantinos Kamnitsas , Mark van der Wilk , Ben Glocker

Several Artificial Intelligence schemes for reasoning under uncertainty explore either explicitly or implicitly asymmetries among probabilities of various states of their uncertain domain models. Even though the correct working of these…

Artificial Intelligence · Computer Science 2013-02-28 Marek J. Druzdzel

Score matching is an alternative to maximum likelihood (ML) for estimating a probability distribution parametrized up to a constant of proportionality. By fitting the ''score'' of the distribution, it sidesteps the need to compute this…

Machine Learning · Computer Science 2023-06-06 Chirag Pabbaraju , Dhruv Rohatgi , Anish Sevekari , Holden Lee , Ankur Moitra , Andrej Risteski

We show how to store good approximations of probability distributions in small space.

Information Theory · Computer Science 2007-07-16 Travis Gagie